>_ IzzyLab

Hands-on task

This task is step 23 of the course “find and xargs: search and bulk operations”

Compress logs in parallel and safely with respect to spaces

medium~5 min

The directory /root/logs contains *.log files, some with spaces in their names (error 1.log). Compress each log with the gzip utility (the output will be *.log.gz files, gzip deletes the originals itself), running the compression in several parallel processes. The pipeline must correctly survive spaces in names.

What you'll practise:
  • find
  • xargs
  • parallelism
  • -print0
  • gzip

Topic breakdown

xargs can not only be safe against spaces (-0 paired with find -print0), but also parallelize work: the -P N flag keeps up to N processes at once, and -n1 hands one argument per command run so the processes are independent. The command here is gzip, which itself replaces each source with its compressed version and deletes the original. This way a batch of files is compressed in parallel and without breaking on names with spaces.

What's checked

  1. all logs compressed
  2. no uncompressed logs remain
  3. the archives are valid

What it looks like