>_ IzzyLab

Hands-on task

This task is step 20 of the course “Processes and signals”

supervisor brings a crashed daemon back

medium~12 min

The queue handler /usr/local/bin/queue-consumer crashes now and then, and there's no systemd in the container — nobody brings it back up. Hand that job to supervisor: describe a program consumer that starts this daemon and restarts it automatically after it dies. The daemon reports its work to stdout — have supervisor store that output in /var/log/consumer.log.

What you'll practise:
  • supervisor
  • supervisorctl
  • auto-restart
  • stdout_logfile
  • daemons

Topic breakdown

A process that died from an error or a signal won't come back by itself — it needs a parent that watches it. In containers without systemd that role is played by supervisor: it starts programs as child processes, gets SIGCHLD when a child dies, and with auto-restart on starts it again. A program is described by a [program:NAME] section in /etc/supervisor/conf.d/; that's also where you say where to keep its stdout — the daemon itself doesn't need to know about log files. supervisor picks up new configs with supervisorctl reread and update.

What's checked

  1. supervisor keeps consumer RUNNING
  2. the daemon's output goes to /var/log/consumer.log
  3. consumer comes back after a crash

What it looks like