gzip
Compressing individual files — gzip, gunzip, zcat, zgrep; a brief look at xz and bzip2
gzip compresses each file separately and replaces it with file.gz. It doesn't pack directories — for that you first build an archive with tar and then compress it (tar -czf). To read compressed files without unpacking there are zcat, zless, zgrep. The .gz format is the most common one: logs are rotated into it, dumps and archives are shipped in it.
Syntax
gzip [options] [file…] # compress: file → file.gz; without files — stdin → stdout
gunzip [options] file.gz… # decompress (same as gzip -d)
zcat file.gz… # print the decompressed contents to stdout
What to do
| Option | What it does |
|---|---|
| (no options) | compress the file and delete the original |
-d | decompress (like gunzip) |
-c | write the result to stdout, leave the source file alone |
-t | test integrity: silent if all is well, code 0 |
-l | show size before and after and the compression ratio |
How to compress
| Option | What it does |
|---|---|
-1 … -9 | level: -1 (--fast) is faster, -9 (--best) compresses more; default -6 |
-k | keep the source file next to the .gz |
-f | overwrite an existing .gz; compress files with hard links and through symbolic links |
-r | process every file in a directory recursively (the directory stays a directory) |
-S .z | use a different suffix instead of .gz (also when decompressing) |
-n / -N | don't save / save and restore on decompression the original file name and time |
--rsyncable | compress so that rsync transfers only the changed parts |
-v / -q | verbose (with compression percentage) / quiet |
gzip -l output
compressed uncompressed ratio uncompressed_name
215163 588895 63.5% access.log
Size after compression, before compression (in bytes), the share saved and the name after decompression. With -v the method, CRC and date are added.
Tools for .gz
| Command | What it does |
|---|---|
zcat | like cat; zcat -f also prints uncompressed files as is |
zless / zmore | page through a compressed file |
zgrep | grep over compressed files, takes the same options (-i, -c, -l, -h) |
zdiff / zcmp | diff / cmp of two compressed files |
xz and bzip2
xz (.xz) compresses noticeably better but slower. bzip2 (.bz2) is an old format between gzip and xz in compression ratio; minimal images often lack it, and "bzip2: not found" means the package isn't installed.
| xz | What it does |
|---|---|
xz file / unxz file.xz | compress / decompress; options -k, -d, -c, -f, -t, -v — as in gzip |
-0 … -9, -e | level; -e — even stronger at the cost of time (-9e) |
-T0 | compress using all cores |
-l / -lv | sizes and compression ratio / details |
xzcat, xzless, xzgrep, xzdiff | counterparts of zcat, zless, zgrep, zdiff |
With tar — the -J option: tar -cJf app.tar.xz ….
Examples
# compress a log; access.log.1 is replaced by access.log.1.gz
gzip /var/log/nginx/access.log.1
# compress, keeping the original, and show the percentage
gzip -kv dump.sql
# decompress
gunzip dump.sql.gz
# search rotated logs without decompressing
zgrep -i 'error' /var/log/syslog.*.gz
zgrep -c ' 500 ' /var/log/nginx/access.log.*.gz
# join the current and compressed logs into one stream
zcat -f /var/log/nginx/access.log* | wc -l
# compress into another file, leave the source alone
gzip -c report.csv > /srv/backup/report.csv.gz
# compress on the fly in a pipeline
pg_dump app | gzip > app.sql.gz
gunzip -c app.sql.gz | head
# test all archives in a directory
for f in /srv/backup/*.gz; do gzip -t "$f" || echo "corrupt: $f"; done
# view the compression ratio
gzip -l dump.sql.gz
# compress every file in a directory separately
gzip -r /srv/app/logs
# xz: compress hard, keep the original, use all cores
xz -k -T0 big.log
xzcat big.log.xz | tail
Pitfalls
- The source file disappears.
gzip filedeletes the original after compressing,gunzipdeletes the.gz. Need the original — add-kor use-cwith a redirect. - A directory isn't compressed: "is a directory -- ignored".
gzip -rcompresses each file inside separately rather than building one archive. For an archive you needtar -czf. - An existing
.gzisn't overwritten: "already exists; not overwritten", exit code 2. Add-f. gunziprequires the extension. A file without.gzgives "unknown suffix -- ignored". Read it withzcat fileorgunzip -c file > result.- Files with hard links and symbolic links are skipped by
gzipwithout-f. With-fon a symbolic link it compresses the target's contents intolink.gzand deletes the link itself. -9isn't always noticeably smaller. On a lot of data the difference from-6is a few percent, while the time grows. For strong compression usexz.zcaton an uncompressed file says "not in gzip format". If the list mixes.logand.log.gz, usezcat -f.- Concatenated
.gzfiles work, but-llies.cat a.gz b.gz > c.gzis a valid file,zcatoutputs both contents. Butgzip -lshows the size of the last part only; the exact size iszcat file | wc -c. - A truncated file.
zcatprints everything it managed to decompress and only then "unexpected end of file". Check the exit code or rungzip -tbefore use.