Blog / Linux

731 words

  • linux
  • bash
  • shell
  • pipefail
  • exit-codes
  • debugging

Why Your CLI's Exit Code Is Always 0 After a Pipe

Your build step failed, the log shows an error, and CI is cheerfully green. Nine times out of ten the culprit is a pipe: make | tee build.log exits 0 even when make does not. The shell is behaving exactly as specified, which is the annoying part.

A pipeline has one exit status, and it is the last command's

By default, the status of a | b | c is the status of c. Everything before it can crash, get killed or exit 99, and the shell throws that information away.

$ false | true
$ echo $?
0

$ ls /nonexistent | wc -l
ls: cannot access '/nonexistent': No such file or directory
0
$ echo $?
0

Here ls failed with status 2, but wc counted zero lines without complaint and exited 0. The error message is on screen; the exit code is not.

Quick fix: set pipefail

In bash, ksh and zsh, set -o pipefail changes the rule. The pipeline's status becomes the rightmost non-zero status of any command in it, or 0 if they all succeeded.

$ set -o pipefail
$ false | true
$ echo $?
1

For scripts, the usual header is:

#!/usr/bin/env bash
set -euo pipefail

Without -e, pipefail only changes the value of $?; nothing stops the script unless you check it. With -e, a failed pipeline now aborts as you probably wanted.

Finding out which stage failed

Pipefail tells you that something failed, not what. Bash keeps every stage's status in the PIPESTATUS array, and it works with or without pipefail.

$ ls /nonexistent | sort | wc -l
$ echo "${PIPESTATUS[@]}"
2 0 0

Copy it immediately. The next command, even an echo, overwrites the array. A safe form is st=("${PIPESTATUS[@]}") on the line straight after the pipeline.

zsh calls it pipestatus (lower case) and indexes from 1 by default. POSIX sh has nothing equivalent, so if you need per-stage detail in plain sh, you are into temporary files or named pipes.

Quick detour: why does the shell only look at the last one?

Because of how the shell waits. It forks every stage, then waits for them. Historically the status of the last wait was the one worth reporting, since the last command is the one producing the final output. Reporting more would have needed a design for combining statuses, and nobody agreed on one for decades.

POSIX has added pipefail in its 2024 edition, so it is on its way to being standard. Whether your particular /bin/sh supports it is another matter: check with sh -c 'set -o pipefail' before relying on it in a #!/bin/sh script.

Where pipefail bites back

Pipefail is not free. The classic trap is a consumer that quits early:

set -o pipefail
yes | head -n 1
echo $?    # 141

head reads one line and exits. yes then writes to a closed pipe and receives SIGPIPE. A process killed by a signal reports 128 plus the signal number, and SIGPIPE is 13, hence 141. The pipeline "failed", though nothing went wrong.

The same thing happens with grep -q:

set -o pipefail
if some_noisy_command | grep -q needle; then
    echo found
fi

Once grep -q finds a match it exits immediately. If some_noisy_command still had output to write, it gets SIGPIPE, the pipeline returns 141, and the if says "not found" even though the needle was there. It is timing dependent too, so it works on your laptop and flakes in CI.

Options for handling it:

  • Drop the early-exit: use grep needle >/dev/null so it reads everything.
  • Capture output first: out=$(some_noisy_command) && grep -q needle <<<"$out".
  • Tolerate 141 explicitly by inspecting PIPESTATUS and treating 141 from the producer as fine.

When you want the failure from a middle stage only

Sometimes a stage is expected to "fail". grep returns 1 for no matches, which is not an error in a filter chain:

set -o pipefail
count=$(cat log | grep ERROR | wc -l)   # aborts under -e if no ERROR lines

Fix it locally with { grep ERROR || true; } in place of the bare grep. That keeps pipefail on for everything else in the script, while this one stage is allowed to find nothing.

Checklist for a pipe you care about

  • Is the exit status used by CI, cron or a calling script? If so, assume the default is hiding failures.
  • Set pipefail, and test it by deliberately breaking an early stage.
  • If any consumer exits early (head, grep -q), expect 141 and handle it.
  • If you need to know which stage broke, read PIPESTATUS straight away.

The nicest test is the dullest one: put false | at the front of the pipeline and confirm the script now fails. If it does not, something upstream is eating the status, usually a subshell or a || true somebody added months ago.

← Back to the blog