>_ IzzyLab

Hands-on task

This task is step 10 of the course “Encodings and checksums”

Twins by sha256

hard~12 min

/srv/images holds firmware images. Exactly two of them have identical contents, even though their names differ (and some others share the same size). Find that pair by SHA-256 checksums and write their names (without the path), one per line, in alphabetical order to /root/twins.txt.

What you'll practise:
  • sha256sum
  • checksums
  • finding duplicates
  • uniq -d
  • comparing contents

Topic breakdown

A checksum is a short "fingerprint" of the contents: identical data always has the same SHA-256, different data practically never does. That's why duplicates are found not by name or size (different files of the same size are common) but by hash. After computing sha256sum for all files, the lines are sorted so equal hashes end up next to each other, and only the repeated ones are kept (uniq -d, comparing just the first 64 characters — the hash itself). What's left is to pick the files with that hash and cut off the path. The same technique is used to deduplicate backups and verify downloaded images.

What's checked

  1. twins found by hash

What it looks like