← Full reference

Redirection and pipelines

Where a command's output goes — >, >>, 2>&1, &>, here-doc, the | pipeline and pipefail

Every program has three standard streams: input (stdin, descriptor 0), output (stdout, 1) and errors (stderr, 2). Redirections replace these streams with files, and the | pipeline connects one command's output to another's input.

Syntax

command > file          # stdout to a file
command 2> file         # stderr to a file
command < file          # stdin from a file
command1 | command2     # stdout of the first — to stdin of the second

Redirections are processed left to right, before the command starts. They can appear anywhere on the line: > f echo hi works too.

Output

OperatorWhat it does
> filestdout to a file; the file is created or truncated to zero
>> filestdout is appended to the end of the file
2> file / 2>> filethe same for stderr
&> file / &>> filestdout and stderr into one file: overwrite / append (bash)
> file 2>&1the same, portable form for sh
>| fileoverwrite even if noclobber is set
> file (no command)create an empty file or truncate an existing one
>/dev/nulldiscard output; 2>/dev/null — discard errors

Input

OperatorWhat it does
< filestdin from a file
<<MARKER … MARKERhere-doc: lines up to the marker go to stdin, $var and $(…) are expanded
<<'MARKER'here-doc without substitutions: text as is
<<-MARKERhere-doc with leading tabs stripped from lines (for indentation in scripts)
<<< "string"here-string: a single string (with an appended \n) to stdin
<(command)process substitution: a command's output as a "file" to read
>(command)the reverse: a "file" whose writes go to the command's stdin

Descriptors

OperatorWhat it does
2>&1stderr to wherever stdout points right now
>&2 (1>&2)stdout to stderr — how scripts write error messages
n>&-close descriptor n
exec 3> fileopen descriptor 3 for writing for the whole script; echo x >&3 writes to the file
exec 3< fileopen descriptor 3 for reading; read -r line <&3
exec 3<> fileopen for reading and writing
exec > file 2>&1redirect all further output of the script
{ …; } > fileredirect the output of a group of commands at once

Pipelines

FormWhat it does
a | bstdout of a goes to stdin of b; both commands run at the same time
a |& bstdout and stderr of a to stdin of b (same as a 2>&1 | b)
${PIPESTATUS[@]}exit codes of all parts of the last pipeline
set -o pipefailthe pipeline's code is the code of the last failed part, not just the last part
! a | binvert the exit code of the whole pipeline

By default a pipeline's exit code is that of the last command: false | true counts as success, an error in the middle is lost. pipefail fixes this:

false | true; echo $?          # 0
set -o pipefail
false | true; echo $?          # 1

Examples

# write output to a file, then append another line
echo one > out.txt
echo two >> out.txt

# errors separately, normal output to the screen
ls /etc/hostname /nope 2> errors.txt

# everything into one file: first stdout to the file, then stderr to the same place
ls /etc/hostname /nope > all.txt 2>&1
ls /etc/hostname /nope &> all.txt          # shorter, bash only

# hide permission errors
find / -name '*.conf' 2>/dev/null

# count lines without printing the file name
wc -l < /etc/passwd

# an error message from a script — to stderr
echo "file not found" >&2

# here-doc: variables inside are expanded
cat <<EOF
home directory: $HOME
EOF

# here-doc with quotes around the marker: text goes as is
cat <<'EOF' > script.sh
echo "$HOME"
EOF

# here-string
tr a-z A-Z <<< "hello"

# send errors into the pipeline along with the output
ls /nope 2>&1 | wc -l

# compare the output of two commands without temporary files
diff <(ls dir1) <(ls dir2)

# read a command's output in a loop without losing variables (no subshell)
while read -r line; do n=$((n+1)); done < <(ls /etc)

# exit codes of each part of a pipeline
false | true; echo "${PIPESTATUS[@]}"      # 1 0

# your own descriptor for a log
exec 3>> /tmp/script.log
echo "step 1" >&3
exec 3>&-

Pitfalls

  • The order of 2>&1 matters. cmd > f 2>&1 writes both streams to the file. cmd 2>&1 > f sends stderr to where stdout pointed before the redirection (the screen), and only stdout goes to the file.
  • sort f > f destroys the file. The shell truncates f before sort starts, and it reads an empty file. Write to a temporary file and rename it (sort has sort -o f f for this).
  • > overwrites silently. set -o noclobber forbids overwriting existing files with >; to force — >|.
  • Each part of a pipeline runs in a subshell. echo "a b" | read x y won't change x in the current shell, and a counter in cat f | while … is empty again after the loop. Use read x y <<< "a b" or while … done < file / < <(command).
  • sudo echo text > /root/file won't work: the redirection is performed by your shell, not by sudo. Use echo text | sudo tee file.
  • <<- strips only tabs, not spaces. The closing marker must be alone on its line, with no spaces after it.
  • &> is bash. In #!/bin/sh (dash) cmd &> f means "run cmd in the background and create an empty f". Write > f 2>&1.

See also

tee · exit-codes · variables · xargs