>_ IzzyLab

Hands-on task

This task is step 15 of the course “Investigating a server”

Where did the inodes go

medium~7 min

Files can no longer be created on the server: No space left on device, even though df -h shows free space. That means the inodes ran out — someone spawned a huge number of small files. The suspect is the application cache /var/lib/webcache: it has the subdirectories thumbs, sessions, logs, uploads (with more nested directories inside).

Find out which subdirectory has the most files (by count, across all nested levels) and write its name, without the path, to /root/inode-hog.txt.

What you'll practise:
  • inode
  • find
  • wc
  • counting files
  • inode exhaustion

Topic breakdown

A filesystem can run out in two ways: by space and by the number of inodes. An inode is the bookkeeping record of a file; their number is fixed when the filesystem is created, and every file, even an empty one, takes one. That's why millions of small files (sessions, thumbnails, temporary chunks) exhaust the inodes while gigabytes are still free: df -h shows space, and df -i shows inodes. The culprit is found by the number of files, not by size: find selecting regular files lists them across the whole subtree, wc -l counts them, and comparing subdirectories points to the source.

What's checked

  1. the culprit subdirectory found

What it looks like