Articles / Linux

787 words

  • linux
  • sigpipe
  • pipes
  • shell
  • rust
  • debugging

Why Your Pipeline Writer Dies Silently with Exit Status 141

Run yes | head -n 1 and you get a single "y" and a prompt. Nothing is printed about yes being killed, yet it was. The kernel sent it SIGPIPE, the default action is to terminate, and the shell stayed quiet about it. The only evidence is an exit status of 141.

This is a different angle from the socket case (Go turns most of those into EPIPE errors). Here the subject is plain pipes, shell pipelines, and what the writer sees when the reader leaves early.

What the kernel does on a write to a dead pipe

When write() targets a pipe with no open read end left anywhere, the kernel does two things:

  • It sends SIGPIPE to the writing process.
  • If that signal is ignored, blocked or caught and returns, write() fails with EPIPE.

The default disposition of SIGPIPE is "terminate". So a C program that never thought about it just vanishes, mid-loop, with no error message. That is the intended design: producer | head should stop the producer rather than let it grind through gigabytes nobody will read.

Seeing the 141

The shell reports a signal death as 128 plus the signal number. SIGPIPE is 13 on Linux, so 141. Bash keeps each stage's status in PIPESTATUS:

$ yes | head -n 1
y
$ echo "${PIPESTATUS[@]}"
141 0

Normally you never see the 141, because a pipeline's status is the last command's. With set -o pipefail it surfaces, and a script that was fine yesterday suddenly "fails" because someone added a head or grep -q downstream. The writer did nothing wrong; the reader was simply satisfied early.

Quick detour: grep -q is a classic trigger. It exits on the first match, closes its end, and the upstream command gets SIGPIPE. Whether that happens depends on timing, so the failure can be intermittent. A short producer that finishes writing before the reader exits never notices anything.

Proving it with strace

If you suspect this and want proof rather than a guess, trace the writer:

$ strace -f -e trace=write,exit_group sh -c 'yes | head -n 1' 2>&1 | tail -n 4

You are looking for a write(1, ...) returning -1 EPIPE (Broken pipe), followed by a --- SIGPIPE line and then +++ killed by SIGPIPE +++. That sequence is unambiguous.

Hang on, why does it sometimes hang instead?

Because "no readers" means no readers anywhere. If some other process inherited the read end of the pipe, perhaps a daemon that was started from the same shell and never closed its descriptors, the pipe is not broken. The writer fills the 64 KiB buffer (the Linux default) and then blocks in write() forever.

So "my producer hangs after the consumer exits" and "my producer dies silently" are the same mechanism with one difference: whether a stray file descriptor kept the read end alive. ls -l /proc/PID/fd on the stuck writer, then looking for the same pipe inode in other processes, finds the culprit.

Rust and Python ignore SIGPIPE on purpose

Both runtimes set SIGPIPE to ignored at startup, so a broken pipe becomes an ordinary error. That is friendlier for long-running programs and much less friendly for command-line filters.

In Rust, println! panics when stdout is a closed pipe: "failed printing to stdout: Broken pipe (os error 32)". Use writeln! and treat BrokenPipe as a normal way to finish:

use std::io::{self, Write};

fn main() -> io::Result<()> {
    let stdout = io::stdout();
    let mut out = io::BufWriter::new(stdout.lock());
    for i in 0u64.. {
        if let Err(e) = writeln!(out, "{i}") {
            if e.kind() == io::ErrorKind::BrokenPipe {
                return Ok(());
            }
            return Err(e);
        }
    }
    Ok(())
}

Note that with a BufWriter the error may only appear when the buffer flushes, so it shows up at a later line than you expect. Also, a clean exit of 0 differs from the 141 a C tool would give; if scripts depend on pipefail, decide which you want and say so in the docs.

The alternative is calling libc::signal(libc::SIGPIPE, libc::SIG_DFL) at the top of main, which needs unsafe and the libc crate, and gives you the traditional Unix behaviour exactly.

Python raises BrokenPipeError, and if it happens during the final flush at interpreter exit you get the noisy "Exception ignored" message. The Python documentation for SIGPIPE handling recommends redirecting stdout to /dev/null before exiting so the final flush cannot fail:

import os, sys

try:
    for i in range(10**9):
        print(i)
    sys.stdout.flush()
except BrokenPipeError:
    devnull = os.open(os.devnull, os.O_WRONLY)
    os.dup2(devnull, sys.stdout.fileno())
    sys.exit(1)

Inherited ignores bite children

An ignored signal disposition survives exec. If a parent starts with SIGPIPE ignored, every child it launches inherits that, and your carefully written C tool will get EPIPE errors it never expected to handle, instead of dying quietly. Python's subprocess defends against this with restore_signals=True, which resets SIGPIPE to default in the child. Not every launcher is so careful.

For sockets, the per-call escape hatch is the MSG_NOSIGNAL flag on send(), which suppresses the signal for that one call. Pipes have no equivalent, so the choice is global: ignore the signal and check every write, or keep the default and let the kernel end the process for you.

← All articles