← Full reference

gzip

Compressing individual files — gzip, gunzip, zcat, zgrep; a brief look at xz and bzip2

gzip compresses each file separately and replaces it with file.gz. It doesn't pack directories — for that you first build an archive with tar and then compress it (tar -czf). To read compressed files without unpacking there are zcat, zless, zgrep. The .gz format is the most common one: logs are rotated into it, dumps and archives are shipped in it.

Syntax

gzip [options] [file…]        # compress: file → file.gz; without files — stdin → stdout
gunzip [options] file.gz…     # decompress (same as gzip -d)
zcat file.gz…               # print the decompressed contents to stdout

What to do

OptionWhat it does
(no options)compress the file and delete the original
-ddecompress (like gunzip)
-cwrite the result to stdout, leave the source file alone
-ttest integrity: silent if all is well, code 0
-lshow size before and after and the compression ratio

How to compress

OptionWhat it does
-1 … -9level: -1 (--fast) is faster, -9 (--best) compresses more; default -6
-kkeep the source file next to the .gz
-foverwrite an existing .gz; compress files with hard links and through symbolic links
-rprocess every file in a directory recursively (the directory stays a directory)
-S .zuse a different suffix instead of .gz (also when decompressing)
-n / -Ndon't save / save and restore on decompression the original file name and time
--rsyncablecompress so that rsync transfers only the changed parts
-v / -qverbose (with compression percentage) / quiet

gzip -l output

         compressed        uncompressed  ratio uncompressed_name
             215163              588895  63.5% access.log

Size after compression, before compression (in bytes), the share saved and the name after decompression. With -v the method, CRC and date are added.

Tools for .gz

CommandWhat it does
zcatlike cat; zcat -f also prints uncompressed files as is
zless / zmorepage through a compressed file
zgrepgrep over compressed files, takes the same options (-i, -c, -l, -h)
zdiff / zcmpdiff / cmp of two compressed files

xz and bzip2

xz (.xz) compresses noticeably better but slower. bzip2 (.bz2) is an old format between gzip and xz in compression ratio; minimal images often lack it, and "bzip2: not found" means the package isn't installed.

xzWhat it does
xz file / unxz file.xzcompress / decompress; options -k, -d, -c, -f, -t, -v — as in gzip
-0 … -9, -elevel; -e — even stronger at the cost of time (-9e)
-T0compress using all cores
-l / -lvsizes and compression ratio / details
xzcat, xzless, xzgrep, xzdiffcounterparts of zcat, zless, zgrep, zdiff

With tar — the -J option: tar -cJf app.tar.xz ….

Examples

# compress a log; access.log.1 is replaced by access.log.1.gz
gzip /var/log/nginx/access.log.1

# compress, keeping the original, and show the percentage
gzip -kv dump.sql

# decompress
gunzip dump.sql.gz

# search rotated logs without decompressing
zgrep -i 'error' /var/log/syslog.*.gz
zgrep -c ' 500 ' /var/log/nginx/access.log.*.gz

# join the current and compressed logs into one stream
zcat -f /var/log/nginx/access.log* | wc -l

# compress into another file, leave the source alone
gzip -c report.csv > /srv/backup/report.csv.gz

# compress on the fly in a pipeline
pg_dump app | gzip > app.sql.gz
gunzip -c app.sql.gz | head

# test all archives in a directory
for f in /srv/backup/*.gz; do gzip -t "$f" || echo "corrupt: $f"; done

# view the compression ratio
gzip -l dump.sql.gz

# compress every file in a directory separately
gzip -r /srv/app/logs

# xz: compress hard, keep the original, use all cores
xz -k -T0 big.log
xzcat big.log.xz | tail

Pitfalls

  • The source file disappears. gzip file deletes the original after compressing, gunzip deletes the .gz. Need the original — add -k or use -c with a redirect.
  • A directory isn't compressed: "is a directory -- ignored". gzip -r compresses each file inside separately rather than building one archive. For an archive you need tar -czf.
  • An existing .gz isn't overwritten: "already exists; not overwritten", exit code 2. Add -f.
  • gunzip requires the extension. A file without .gz gives "unknown suffix -- ignored". Read it with zcat file or gunzip -c file > result.
  • Files with hard links and symbolic links are skipped by gzip without -f. With -f on a symbolic link it compresses the target's contents into link.gz and deletes the link itself.
  • -9 isn't always noticeably smaller. On a lot of data the difference from -6 is a few percent, while the time grows. For strong compression use xz.
  • zcat on an uncompressed file says "not in gzip format". If the list mixes .log and .log.gz, use zcat -f.
  • Concatenated .gz files work, but -l lies. cat a.gz b.gz > c.gz is a valid file, zcat outputs both contents. But gzip -l shows the size of the last part only; the exact size is zcat file | wc -c.
  • A truncated file. zcat prints everything it managed to decompress and only then "unexpected end of file". Check the exit code or run gzip -t before use.

See also

tar · logrotate · grep · rsync · checksums · pg-tools