curl Without Surprises: Save, Redirect and Check Status

By default curl will happily save a 404 error page and call it a success. Here is how to make curl fail loudly, save safely and follow redirects on purpose. You will build a small, repeatable workflow in about fifteen minutes.

• You need: a shell, curl, and a network route to the example URLs.
• No root: these are ordinary user operations; none of them need sudo.
• Tested on: curl 8.5.0 from Ubuntu package version 8.5.0-2ubuntu10.13.
• Your data: replace the example URL and file names with values you trust, and keep the original file when the download matters.

1. Check which curl you have

Find out which binary and version your shell will actually run:

$ command -v curl
/usr/bin/curl
$ curl --version | sed -n '1,2p'
curl 8.5.0 (x86_64-pc-linux-gnu) libcurl/8.5.0 OpenSSL/3.0.13 ...
Release-Date: 2023-12-06, security patched: 8.5.0-2ubuntu10.13

The feature list depends on the build. This one supports HTTP and HTTPS, among other protocols. Another machine's curl may not share the same protocol support or option behaviour just because it has the same name.

2. Look before you download

With no output option, curl writes the body to standard output. Fine for a bit of text, a poor default for anything large or binary.

When you only need the headers, use -I (also spelled --head):

$ curl --head https://example.com
HTTP/2 200
content-type: text/html
...

• What to check: the status line, plus fields such as content-type or content-length. Header order and values vary.
• What HEAD is not: a substitute for downloading. Some servers handle HEAD differently or leave out fields you need.

For a compact result with no body, throw the body away and ask curl to report selected transfer details:

$ curl --silent --show-error --output /dev/null \
    --write-out 'status=%{response_code} effective=%{url_effective}\n' \
    https://example.com
status=200 effective=https://example.com/

• --silent hides the progress meter.
• --show-error keeps diagnostics visible when the transfer fails.
• --write-out output is separate from the response body, which makes this pattern good for checks and scripts.

3. Save to a new path

Use --output to write the response to a named file:

$ curl --fail --silent --show-error \
    --output response.html \
    https://example.com/
$ test -s response.html && echo 'saved a non-empty response'
saved a non-empty response

--fail turns HTTP responses of 400 and above into exit status 22, instead of treating the error page as a good download. It does not validate the document's contents, and it does not give every transport problem the same code. Always check the exit status in automation.

$ curl --fail --silent --show-error \
    --output response.html.new \
    https://example.com/
$ test -s response.html.new
$ mv response.html.new response.html

4. Follow redirects on purpose

curl does not follow HTTP redirects by default. Add --location when the URL may redirect, such as a download link that points at the current version:

$ curl --fail --silent --show-error --location \
    --output response.html \
    https://example.com/
$ curl --silent --show-error --output /dev/null \
    --write-out 'status=%{response_code} redirects=%{num_redirects} effective=%{url_effective}\n' \
    --location https://example.com
status=200 redirects=0 effective=https://example.com/

On a redirecting URL, num_redirects counts the redirects followed and url_effective shows where you ended up.

5. Make scripts fail clearly

A script should never wait forever. Bound the request:

• --connect-timeout limits the connection phase.
• --max-time limits the whole operation.

$ curl --fail --silent --show-error \
    --connect-timeout 5 --max-time 20 \
    --output response.html \
    https://example.com/
$ printf 'curl exit status: %s\n' "$?"
curl exit status: 0

Do not print a success message before checking the exit status. In a script, make the check part of the same decision:

if curl --fail --silent --show-error --location \
    --connect-timeout 5 --max-time 20 \
    --output response.html.new https://example.com/; then
    test -s response.html.new && mv response.html.new response.html
    printf '%s\n' 'download completed'
else
    status=$?
    rm -f response.html.new
    printf 'curl failed with exit status %s\n' "$status" >&2
    exit "$status"
fi

On failure it removes only its own temporary path and leaves the previous response.html alone.

Exit statuses worth knowing:

• 6: could not resolve the host.
• 7: failed to connect.
• 22: HTTP error, when --fail is used.
• 28: timeout.

6. Debug without switching off TLS

When a transfer fails, add --verbose for request and connection detail:

$ curl --verbose --fail --output /dev/null https://example.com
* Connected to example.com (...)
> GET / HTTP/2
< HTTP/2 200
...

Addresses, headers and server details will vary.

Check these instead:

• The hostname. Is it the one on the certificate?
• The system clock.
• The proxy.
• The CA configuration. If a private service needs its own trust anchor, pass a deliberately chosen CA file with --cacert and protect that file.

These are security-sensitive choices. curl itself needs no root for them, but installing a system-wide CA may need an administrator and should follow your distribution's process.

7. Keep request data separate from downloads

For a form or API request, --data sends data rather than just fetching a page.

• Keep credentials out of command history and process listings where you can.
• Never copy a real token into an example or commit it to a script.
• Test against a non-production endpoint and confirm exactly what is being sent. The local manual documents plenty of authentication and upload options; that is the safe default.

Quote URLs containing shell metacharacters such as &, ? or *. curl's own URL globbing uses braces and brackets, and the shell can interpret several of the same characters:

$ curl --fail --silent --show-error \
    --output search.html \
    'https://example.com/search?q=linux&page=1'