sed
Stream editor — regex substitution, deleting and inserting lines, editing files in place
sed reads text line by line, applies the script's commands to each line and prints the result. It's most often used for the s/old/new/ substitution, but it can also delete, insert and select lines by number or pattern. The sandbox has GNU sed 4.9.
Syntax
sed [options] 'script' [file…]
sed [options] -e 'command1' -e 'command2' [file…]
sed [options] -f script.sed [file…]
How it works: sed puts the next line (without \n) into a buffer — the pattern space, runs all the script's commands on it in order, prints the buffer (unless -n) and takes the next line. Commands in one script are separated by ; or a newline. The original file isn't changed unless -i is given.
Options
| Option | What it does |
|---|---|
-n | don't print the buffer automatically — only what p, P, = etc. print is output |
-e script | add commands; can be repeated |
-f file | take commands from a file |
-E (-r) | extended regexes: + ? | ( ) { } without backslashes |
-i | edit files in place |
-i.bak | edit in place, keeping the original as file.bak (the suffix is written without a space) |
--follow-symlinks | with -i: edit the file a link points to instead of replacing the link itself |
-s | treat files separately: $ and line numbers are per file (enabled automatically with -i) |
-z | lines are separated by a NUL byte, not \n — so a whole file without NULs becomes one "line" |
-u | unbuffered output (for tail -f | sed …) |
--posix | disable GNU extensions |
--debug | show how the script is executed |
--sandbox | forbid the e, r, w commands that work with files and the shell |
Addresses
An address before a command limits the lines it applies to.
| Address | Which lines |
|---|---|
| (none) | all |
5 / $ | the fifth / the last |
/re/ | lines with a match; \%re% — the same with another delimiter; /re/I — case-insensitive |
2,5 | 2nd through 5th |
/start/,/end/ | from a line with start to the nearest following line with end, inclusive; the range can repeat |
5,$ / /re/,$ | from the 5th (or from a match) to the end |
0,/re/ | from the beginning to the first match, even if it's on the first line (GNU) |
/re/,+2 | the matching line and 2 more after it (GNU) |
1~3 | every third line, starting with the first (GNU) |
address! | all lines except those matching the address |
Commands
| Command | What it does |
|---|---|
s/re/replacement/flags | replace a match (details below) |
d | delete the buffer and move on to the next line immediately |
p | print the buffer (with -n — the only output) |
= | print the line number |
a text / i text | insert a line after / before the current one |
c text | replace the line (or the whole range) with text |
y/abc/xyz/ | character-by-character replacement, like tr |
q / Q | quit, printing / not printing the current line; q5 — with exit code 5 |
r file / w file | insert a file's contents after the line / append the line to a file |
n / N | replace the buffer with the next line / append the next line to the buffer with \n |
h H g G x | copy / append the buffer to the spare one (hold space), get / append from there, exchange |
{ …; … } | a group of commands under one address |
The s command
The delimiter is any character right after s: s|/a|/b|, s#x#y#.
| Element | Meaning |
|---|---|
& in the replacement | the whole match |
\1 … \9 | groups from parentheses (\( \) in BRE, ( ) with -E) |
\n in the replacement | newline |
\U \L / \u \l / \E | everything after in upper / lower case, the next character in upper / lower case, end of conversion (GNU) |
flag g | replace all occurrences in the line, not just the first |
flag N (a number) | replace only the N-th occurrence; Ng — the N-th and all after it |
flag I | case-insensitive |
flag p | print the line if a replacement was made (usually with -n) |
flag w file | append the changed line to a file |
Examples
printf 'one\ntwo\nthree\nfour\nfive\n' > n.txt
# the first "o" in each line; with g — all; only the second occurrence
sed 's/o/0/' n.txt
sed 's/o/0/g' n.txt
echo 'foo foo' | sed 's/o/0/2'
# replace only in lines containing "t"
sed '/t/s/e/E/g' n.txt
# print lines 2–4; the matching line and one after it
sed -n '2,4p' n.txt
sed -n '/two/,+1p' n.txt
# delete lines starting with t; delete empty lines and lines of only spaces
sed '/^t/d' n.txt
printf 'a\n\n \nb\n' | sed '/^[[:space:]]*$/d'
# delete everything except lines with "o"
sed '/o/!d' n.txt
# another delimiter when the text has lots of slashes
echo '/usr/local/bin' | sed 's|/usr/local|/opt|'
# rearrange parts via groups; & — the whole match
echo 'key = value' | sed -E 's/^([a-z]+) = (.*)/\2=\1/'
echo 'abc' | sed 's/b/[&]/'
# case
echo 'hello world' | sed -E 's/\w+/\u&/g'
echo 'hello' | sed 's/.*/\U&/'
# comment a line out and uncomment it back
echo 'option=1' | sed 's/^/# /'
echo '# option=1' | sed 's/^#[[:space:]]*//'
# insert, replace a line, line number
sed '2i --- before two ---' n.txt
sed '/three/c 3' n.txt
sed -n '/four/=' n.txt
# in-place edit with a backup; several commands
sed -i.bak -e '1i header' -e '$a footer' n.txt
# first 3 lines and quit (without reading the rest of the file)
sed 3q n.txt
# join all lines into one, comma-separated
printf 'a\nb\nc\n' | sed -z 's/\n/,/g; s/,$/\n/'
# substitute a shell variable: double quotes and a delimiter that isn't in the value
dir=/srv/data
echo 'path=old' | sed "s|old|$dir|"
Pitfalls
- Without
-E, parentheses and+are ordinary characters.sed 's/(a+)/X/'searches for a literal(a+). Either add-Eor escape them:\(a\+\). - Single quotes don't let variables through.
sed 's/x/$var/'inserts the text$var. Use double quotes — and make sure$and`in the script itself aren't eaten by the shell. - Slashes in the value break the command. If the substituted string contains
/, use another delimiter:s|…|…|ors#…#…#.&and\in the value also need escaping. -iwithout a backup is irreversible. Run without-ifirst and look at the output. The suffix is written without a space:-i.bak; sed takes-i .bakas the script.bak. On macOS (BSD sed) the syntax differs:-i ''.-ireplaces a symbolic link with a regular file and leaves the original file untouched. To edit the link's target, add--follow-symlinks.-n+p. Without-nthepcommand prints the line twice. And-nwithoutpprints nothing.sedworks line by line:\nwon't appear in the pattern until you join lines yourself withNor-z.- Regexes are greedy.
s/X.*X/-/grabs from the first to the lastXin the line. sed has no lazy.*?— use a negated class:X[^X]*X. - The range
/a/,/b/looks for the end starting from the next line. Ifbisn't found, the range lasts until the end of the file.
See also
Regular expressions · grep · awk · tr