>_ IzzyLab

Hands-on task

This task is step 20 of the course “awk and datamash: reports from tables”

awk: join two files by key

hard~10 min

There are two files with a shared key in the first field: /root/names.txt (id name) and /root/ages.txt (id age). Write an awk program in the file /root/join.awk (handy to do in the Editor tab above the terminal): it gets two such files — first id age, then id name — and prints lines name age in the order of the second file, only for ids that have both a name and an age. Run it with awk -f on ages.txt and names.txt and write the result to /root/joined.txt (ids in ascending order there). The program will also be tested on other files.

What you'll practise:
  • awk
  • join by key
  • NR and FNR
  • associative arrays
  • next

Topic breakdown

Joining two files by a shared key is built on two of awk's built-in variables: NR counts lines cumulatively across all input files, while FNR counts only within the current one, resetting for each new file. Their equality is true exactly while the first file is being read, which lets you load a "key→value" lookup into an associative array on the first pass and, on the second file's lines, fetch matches from it by key (the key in array test drops lines without a pair). The second file sets the output order, so the lookup is built from the file whose order doesn't matter. next prevents the first file's rules from firing on the second. When there's more than one rule, it's handier to keep the program in a separate file and run it with awk -f: you can read it with indentation and comments, and the shell's quotes don't get in the way.

What's checked

  1. the files are joined by key
  2. join.awk works on other files

What it looks like