← Full reference

Regular expressions

BRE, ERE and PCRE in practice — classes, anchors, quantifiers, groups, backreferences, greediness

A regular expression is a pattern describing a set of strings. grep, sed, awk and many other programs understand them, but in several dialects. The basics are shared; the differences are mostly in which special characters need (or don't need) escaping.

Three dialects

  • BRE (basic) — grep and sed without options. The characters + ? | ( ) { } are ordinary here; backslashes give them special meaning: \+ \? \| \( \) \{ \} (\+ \? \| are a GNU extension).
  • ERE (extended) — grep -E, sed -E, awk. The same + ? | ( ) { } work without backslashes, while \+ and \( on the contrary mean ordinary characters.
  • PCRE (Perl) — grep -P. Everything from ERE plus \d \s \w, lazy quantifiers *?, lookarounds (?=…) and (?<=…), \K, modifiers (?i).

The same thing in three dialects — "one or more digits, then a dot":

echo 'v12.3' | grep -o '[0-9]\+\.'      # BRE
echo 'v12.3' | grep -oE '[0-9]+\.'      # ERE
echo 'v12.3' | grep -oP '\d+\.'         # PCRE

Characters and sets

ElementMeaning
a, 1, -an ordinary character matches itself
.any single character
\. \* \[ \\ \$ \^a special character as an ordinary one
[abc]one character from the set
[^abc]any character except those listed
[a-z] [0-9A-F]ranges inside a set
[]a] / [a-] / [.*]] — first, - — first or last; . and * inside brackets are ordinary characters

POSIX classes

Written inside square brackets: [[:digit:]], [^[:space:]], [[:alpha:]_].

ClassWhat it includes
[:digit:] / [:xdigit:]digits / hexadecimal digits
[:alpha:] / [:alnum:]letters / letters and digits
[:upper:] / [:lower:]upper case / lower case
[:space:] / [:blank:]all whitespace (space, \t, \n, \r …) / only space and tab
[:punct:]punctuation
[:print:] / [:graph:] / [:cntrl:]printable including space / printable without space / control characters

Shorthands

ElementMeaningWhere it works
\w / \Wword character (letter, digit, _) / not oneGNU grep, sed, gawk, PCRE
\s / \Swhitespace / non-whitespaceGNU grep, sed, gawk, PCRE
\d / \Ddigit / non-digitPCRE only (grep -P)
\t \ntab, newlinePCRE, gawk; in sed \n and \t too

Anchors

ElementMeaning
^ / $start / end of line
\< / \>start / end of a word (GNU grep, sed, gawk)
\b / \Bword boundary / not a boundary (GNU grep, sed, PCRE; in gawk the boundary is \y)

Quantifiers

They apply to the preceding character, set or group.

ERE / PCREBREHow many times
**0 or more
+\+1 or more
?\?0 or 1
{3}\{3\}exactly 3
{2,5}\{2,5\}from 2 to 5
{2,} / {,5}\{2,\} / \{,5\}2 or more / at most 5

All quantifiers are greedy: they take as much as possible and then back off if a match can't be found otherwise. PCRE has lazy *?, +?, ??, {2,5}? — they take as little as possible.

Groups and alternation

ElementMeaning
(…) (in BRE — \(…\))group: a quantifier applies to all of its content, the matched text is remembered
\1 … \9backreference — the same text the group matched. In grep and sed patterns, in sed replacements; awk doesn't have them
a|ba or b; inside a group — (jpg|png). In BRE the bar needs a backslash
(?:…)non-capturing group (PCRE)
(?=…) / (?!…)followed / not followed by … (PCRE, not part of the match)
(?<=…) / (?<!…)preceded / not preceded by … (PCRE)
\Kforget everything matched up to this point (PCRE)
(?i)case-insensitive from here on (PCRE)

Examples

# a line consisting entirely of digits
printf '123\n12a\n' | grep -E '^[0-9]+$'

# empty lines and lines of only spaces
printf 'a\n\n  \nb\n' | grep -c '^[[:space:]]*$'

# lines whose first character is neither a space nor #
printf '# comment\n  indented\nvalue\n' | grep '^[^[:space:]#]'

# a whole word: cat, but not catalog
echo 'cat catalog' | grep -o '\<cat\>'
echo 'cat catalog' | grep -oE '\bcat\b'

# an optional letter and an exact number of repeats
echo 'color colour' | grep -oE 'colou?r'
echo '2024-01-05' | grep -E '^[0-9]{4}-[0-9]{2}-[0-9]{2}$'

# alternation and a group with a quantifier
printf 'ab\nabab\naba\n' | grep -xE '(ab)+'
printf 'a.jpg\nb.png\nc.txt\n' | grep -E '\.(jpg|png)$'

# backreference: two identical characters in a row
echo 'aa bc dd' | grep -oE '(.)\1'

# groups in a replacement: rearrange the parts of a date
echo '2024-01-05' | sed -E 's/([0-9]{4})-([0-9]{2})-([0-9]{2})/\3.\2.\1/'

# greediness: .* grabs the maximum
echo '<a><b>' | grep -oE '<.*>'
# the lazy variant exists only in PCRE
echo '<a><b>' | grep -oP '<.*?>'
# a portable substitute for laziness — a negated class
echo '<a><b>' | grep -oE '<[^>]*>'

# PCRE: lookbehind and \K — a match without the prefix
echo 'id=42' | grep -oP '(?<=id=)\d+'
echo 'id=42' | grep -oP 'id=\K\d+'

# PCRE: a word NOT followed by a digit
echo 'v1 vx' | grep -oP 'v(?!\d)\w'

# a special character as an ordinary one: dot and asterisk
printf '1.5\n105\n' | grep '^1\.5$'
printf 'a*b\naab\n' | grep 'a\*b'

Pitfalls

  • A dot is any character. grep '1.2' also finds 1x2. Escape it (1\.2), put it in brackets (1[.]2) or use grep -F to search for a literal string.
  • POSIX classes only inside brackets. [:digit:] without the second pair of brackets means the set of characters : d i g t; GNU grep stops with the error character class syntax is [[:space:]], not [:space:] in this case.
  • \d outside -P is the letter d, and \t in grep is the letter t. Write [0-9] or [[:digit:]], and pass a tab as $'\t'.
  • No lazy quantifiers in BRE/ERE. .*? in grep -E and sed -E works like the usual greedy .*. Use [^…]* or grep -P.
  • ^ and $ in BRE are special only at the edges. grep 'a^b' searches for a literal a^b.
  • * at the start of a pattern is an ordinary character in BRE: grep '*a' searches for *a. In ERE GNU grep accepts it with the warning * at start of expression.
  • A regex searches for a substring. grep '[0-9]' finds the line abc1. To check the whole line, use ^…$ or grep -x.
  • Quote the pattern. Without single quotes the shell processes *, ?, […], $ and \ itself.
  • Ranges and . depend on the locale. The sandbox locale is POSIX: . is one byte, [a-z] is exactly the Latin lower-case letters, and [[:alpha:]] doesn't include accented letters. With LC_ALL=C.utf8 . matches a whole letter, and [[:alpha:]] finds é.

See also

grep · sed · awk