>_ IzzyLab

Hands-on task

This task is step 21 of the course “Processes and signals”

Two daemons and a graceful stop

medium~12 min

Two programs must run under supervisor: ingest (/usr/local/bin/ingestd) and notifier (/usr/local/bin/notifierd), both restarted automatically after a crash.

There's a catch: notifierd shuts down cleanly (flushes its queue and leaves the marker /var/tmp/notifier.graceful) only on the INT signal, while on TERM, which supervisor sends by default, it simply dies. Make supervisorctl stop notifier stop it with INT.

What you'll practise:
  • supervisor
  • stopsignal
  • SIGINT
  • auto-restart
  • graceful shutdown

Topic breakdown

A process manager stops a program with a signal, and by default that's SIGTERM. But daemons read signals differently: some shut down cleanly on TERM, others only on INT or QUIT, and TERM means instant death without cleanup for them. That's why in supervisor the stop signal is configured per program, together with the wait before SIGKILL. Each program is described by its own [program:NAME] section; after editing the configs, they're picked up with supervisorctl reread and update.

What's checked

  1. ingest is RUNNING
  2. notifier is RUNNING
  3. both programs come back after a crash
  4. notifier is stopped with the INT signal

What it looks like