← Full reference

wget

Downloading files over HTTP/HTTPS — resuming, link checking, recursive download, exit codes

wget downloads files by URL and by default saves them to disk under the name from the address. It can resume interrupted downloads, follow links recursively and follows redirects on its own. For a one-off "download this file" it's simpler than curl; for API requests with methods and headers curl is handier.

Syntax

wget [options] URL [URL…]

The work log (Connecting…, 200 OK, progress) goes to stderr, the body — to the file.

Options

Where to save

OptionWhat it does
-O filesave to this file; -O - — to stdout
-P directorysave into a directory (the current one by default)
-ncdon't download if the file already exists (otherwise you get name.1)
-Ndownload only if the file on the server is newer than the local one
-cresume a partially downloaded file (the server must support Range)
--content-dispositiontake the file name from the Content-Disposition header

What to output

OptionWhat it does
-qquiet: no log
-nvbrief: one line per file
-Sshow the server's response headers
-o file / -a filewrite the log to a file / append to a file
--spiderdownload nothing, just check that the URL exists

The request

OptionWhat it does
--header 'Name: value'add a header; can be repeated
-U stringa custom User-Agent
--user u --password pHTTP authentication (Basic/Digest)
--post-data 'a=1&b=2' / --post-file filesend a POST
--no-check-certificatedon't verify the TLS certificate
--max-redirect=Nat most N redirects (0 — don't follow them)
-T sectimeout for connecting and reading
-t Nnumber of attempts (20 by default; 0 — infinite)
--limit-rate=200klimit the speed
-i filetake the list of URLs from a file (one per line)

Recursive download

OptionWhat it does
-rrecursively follow links from HTML
-l Nrecursion depth (5 by default)
-npdon't ascend above the starting directory
-nddon't create directories — all files into one folder
-nHdon't create a directory named after the host
-A patterns / -R patternsdownload only / don't download files by suffix or pattern: -A 'pdf,txt'
-krewrite links in downloaded HTML to local ones
-palso download everything needed to display the page (images, CSS)
-mmirror: same as -r -N -l inf --no-remove-listing

Exit codes

CodeMeaning
0all good
1generic error
2option parsing error
3file write error
4network error: couldn't connect, timeout, name doesn't resolve
5TLS certificate verification failed
6authentication error (401)
7protocol error
8the server responded with an error: 404, 500…

wget or curl

Taskwgetcurl
save a fileby defaultneeds -O or -o
follow redirectsby defaultneeds -L
HTTP error → non-zero codeby default (8)needs -f
download a site recursively-rcan't
PUT/DELETE methods, arbitrary body, JSON APIsawkward-X, -d, -H

Examples

# download a file under its own name
wget http://localhost:8080/backup.tar.gz

# into a specific file, quietly
wget -q -O /tmp/page.html http://localhost:8080/

# to stdout — for a pipeline
wget -qO- http://localhost:8080/api/status | jq .

# only the response headers, discard the body
wget -S -q -O /dev/null http://localhost:8080/

# check whether the URL exists (code 0) or not (code 8)
wget -q --spider http://localhost:8080/health && echo ok

# resume an interrupted download
wget -c http://localhost:8080/big.iso

# several files from a list, into the downloads directory
wget -q -i urls.txt -P downloads

# download all .txt files the page links to, without a directory tree
wget -r -l 1 -nd -A txt http://localhost:8080/index.html

# a quick timeout and a single attempt — for scripts
wget -q -T 5 -t 1 http://10.0.0.5/ || echo "code $?"

# Basic authentication and a custom header
wget -qO- --user admin --password secret --header 'Accept: application/json' http://localhost:8080/private/

Pitfalls

  • -O creates the file even on error. wget -O out.html http://…/nope gets a 404 and leaves an empty out.html. Check the exit code, not whether the file exists.
  • Running it again doesn't overwrite the file, it creates name.1, name.2… To overwrite, use -O name; to avoid re-downloading — -nc or -N.
  • -c requires Range support on the server. If the server doesn't support it (e.g. python3 -m http.server), wget downloads the whole file again.
  • -O with several URLs concatenates them into one file.
  • -t defaults to 20 attempts. "Connection refused" and 404 aren't retried, but timeouts and dropped connections are — up to 20 times, so wget against an unreachable address can hang for minutes. In scripts, set -T and -t 1.
  • --spider still connects to the server — it's not an offline check; with -r it walks the links but doesn't save files.
  • The log goes to stderr. wget … | grep on the log won't work: add 2>&1 first.
  • -A still downloads HTML with -r to find links in it, and deletes it afterwards.

See also

curl · htpasswd · nginx · ab · python3