>_ IzzyLab

Hands-on task

This task is step 15 of the course “JSON, YAML and XML in the terminal”

jq: JSONL to a CSV summary

hard~12 min

/root/events.jsonl — one JSON object {user, action} per line. Write a jq filter in the file /root/summary.jq (handy to do in the Editor tab above the terminal) that counts the number of events per user. The filter must work when run as jq -s -r -f /root/summary.jq <file> on any file of this format, not just this one. Write the result for events.jsonl to /root/summary.csv: a CSV with the header user,count, sorted by user. The format is what @csv produces (strings in quotes, numbers without).

What you'll practise:
  • jq
  • jsonl
  • group_by
  • @csv
  • aggregation and sorting

Topic breakdown

JSONL is a stream of separate objects one per line, not a single array, so it is first "slurped" with the slurp flag -s into an array. Grouping by a field is done with the group_by(...) function, which lays elements out into an array of groups; each group is collapsed into a single record, where the counter is given by the built-in length and the common field value is taken from any element of the group. For CSV output, a stream of array-lines is passed through the @csv filter (which places quotes by CSV rules itself) with the raw-output flag -r; the header is added as a separate first line. A long filter is kept in a separate .jq file and loaded with the -f option: in a file you can split it into lines by pipeline step, and the shell's quotes no longer get in the way.

What's checked

  1. per-user CSV summary is correct
  2. summary.jq works on other data

What it looks like