>_ IzzyLab

Hands-on task

This task is step 17 of the course “Troubleshooting and recovering a Linux server”

The log is filling up the disk

medium~7 min

The file /var/log/app/app.log is growing uncontrollably (it's already large). Urgently truncate it to zero without deleting the file itself (the service holds an open descriptor on it), and set things up for the future — create a logrotate config /etc/logrotate.d/app with size-based rotation.

What you'll practise:
  • file descriptors
  • truncate
  • logrotate
  • inode
  • log management

Topic breakdown

When a service holds an open descriptor on a log file, it's important to free space without touching the inode itself: if you rm and create a new file with the same name, the process keeps writing to the old deleted inode and the space won't come back until a restart. The correct technique is to truncate the contents of the same file to zero (the truncate utility or redirection in overwrite mode), keeping the descriptor alive. Separately you configure logrotate: a config in /etc/logrotate.d/ describes the log path and a block of directives — a rotation threshold by size (size), the number of kept copies (rotate), compression; for processes with an open descriptor, real life often adds copytruncate.

What's checked

  1. Log truncated
  2. File not deleted
  3. The file was truncated, not recreated
  4. The service still writes to app.log
  5. logrotate is set up for size-based rotation

What it looks like