yq
Reading and editing YAML jq-style — selection, select, in-place editing, merging files, converting to JSON
yq (the mikefarah version, v4) is jq for YAML: the same expression language, but it reads and writes YAML, as well as JSON, XML, TOML, CSV and properties. The main differences from jq: it can edit a file in place (-i) and it preserves comments and formatting.
Syntax
yq [options] 'expression' [file…]
yq eval 'expression' file # the same: eval (e) is the default command
yq eval-all 'expression' file… # (ea) all files and documents at once, as one stream
Without a file it reads stdin. Put the expression in single quotes.
Options
| Option | What it does |
|---|---|
-i | write the result back into the (first) file |
-o=FORMAT | output format: yaml, json, xml, props, csv, tsv, shell (TOML output in v4.44 is supported only for simple values) |
-p=FORMAT | input format: yaml, json, xml, props, csv, tsv, toml; guessed from the extension by default |
-I=N | indent; -o=json -I=0 — JSON on one line |
-P | "pretty" YAML: block style instead of {…} and […] |
-r | print strings without quotes and comments — on by default for YAML; -r=false — turn it off |
-e | exit code 1 if the result is null, false or nothing was found |
-n | don't read input — create a document from scratch |
-N | don't print the --- document separators |
-C / -M | force color on / off |
--from-file file | take the expression from a file |
-f extract | work with the YAML front matter of a markdown file |
-s 'expression' | split results into files, the file name comes from the expression |
There are no --arg flags like in jq: pass values from the shell via the environment (strenv, env).
Paths and selection
| Expression | What it does |
|---|---|
.a.b, .a[0], .a[-1], .a[] | field, element, last element, all elements |
.["a.b"] | a key with a dot or special characters |
.. | all nodes recursively |
select(.name == "x") | filter by condition; and, or, not are available |
has("k"), keys, length | key presence, list of keys, length |
tag | node type: !!str, !!int, !!bool, !!map, !!seq, !!null |
path, key, parent | path to the node, its key or index, the parent node |
a // b | b if a is null, false or nothing |
select(di == 1) | the second document of a file with several --- (di = document_index) |
Modification
| Expression | What it does |
|---|---|
.a = "v" | assign; intermediate keys are created automatically |
.a |= f | update a value with an expression: .name |= upcase |
(… | select(…) | .f) = "v" | change a field only in matching elements, keeping the whole document |
.list += ["x"] | append to an array |
del(.a) / del(.list[] | select(…)) | delete a field / matching elements |
. * {"a": 1} | deep merge of objects (the right one wins); *+ — arrays are concatenated |
.a style="double" | value style: double, single, literal (|), folded (>), flow |
... comments = "" | remove all comments |
explode(.) | expand anchors and aliases (&a, *a, <<:) |
sort_keys(..) | sort keys at all levels |
Functions
| Functions | What they do |
|---|---|
strenv(VAR) / env(VAR) | an environment variable's value: always as a string / with type parsing |
envsubst | substitute ${VAR} inside a string |
load("file") | load another YAML file |
map(f), to_entries, from_entries, with_entries(f) | as in jq |
sort, sort_by(f), unique, group_by(f), reverse | sorting and grouping |
split(s), join(s), upcase, downcase, test(re), sub(re; s) | strings and regexes |
line_comment, head_comment, foot_comment | read or set a node's comment |
@json, @yaml, @base64, @base64d | encode a value into a string of another format |
Examples
cat > app.yaml <<'EOF'
app:
name: demo
replicas: 2
env:
- name: MODE
value: dev
EOF
# read a value
yq '.app.replicas' app.yaml
# find a list element by a field
yq '.app.env[] | select(.name == "MODE") | .value' app.yaml
# show a change without touching the file
yq '.app.replicas = 3' app.yaml
# change in place
yq -i '.app.replicas = 3' app.yaml
# change the value of a specific list element
yq -i '(.app.env[] | select(.name == "MODE") | .value) = "test"' app.yaml
# add a nested field — intermediate keys are created automatically
yq -i '.app.labels.tier = "web"' app.yaml
# a value from an environment variable, as a string
NAME=demo2 yq '.app.name = strenv(NAME)' app.yaml
# YAML → JSON and on into jq
yq -o=json '.app' app.yaml | jq -r '.name'
# JSON → YAML
echo '{"k":[1,2]}' | yq -p=json -o=yaml
# merge two files: values from override.yaml override base.yaml
yq ea '. as $item ireduce ({}; . * $item)' base.yaml override.yaml
# all values as a flat "path = value" list
yq -o=props app.yaml
# create a document from scratch
yq -n '.server.port = 8080'
Pitfalls
- There are two different
yqs. The Python version (kislyuk/yq) is a wrapper aroundjqwith different flags (e.g.-yfor YAML output) and no comment preservation. Checkyq --version: you need the linemikefarah/yq … v4. - Without
-ithe file doesn't change — the result is just printed. Andyq '…' f.yaml > f.yamlempties the file, as with any command. - An assignment after
|changes the whole output..app.env[] | select(…) | .value = "x"outputs only the selected elements. To get the whole file with the change, put the left side in parentheses:(… | .value) = "x". - Types matter.
.version = 1.10writes a number, and.version = "1.10"— a string. For values from the environment:strenv()always gives a string,env()parses the type (5becomes a number). - A nonexistent path gives
null, not an error. For checks add-e. - Several documents in a file. The expression is applied to each document separately, and the output is separated by
---. To work with them together (collect into an array, merge), you needeval-all. *merges maps but replaces arrays entirely. To concatenate lists, use*+.- YAML treats
"false"in quotes as a string andfalsewithout quotes as a boolean; the same goes foryes/no,on/offin other programs' YAML 1.1 parsers. If a value must stay a string, keep the quotes.
See also
jq · kubectl · ansible · docker-compose · gitlab-ci