Articles / Linux

736 words

  • linux
  • pipes
  • shell
  • backpressure
  • systems-programming
  • debugging

Why a Unix Pipe Blocks the Writer When the Reader Is Slow

A pipe is not a stream that flows freely between two programs. It is a small fixed-size buffer inside the kernel, and when it fills, the writer is put to sleep until the reader makes room. That is the whole story, and it is also why a fast producer feeding a slow consumer appears to "hang" at exactly the same point every time.

Watch it stall

Here is a writer that would happily push data forever, and a reader that does nothing for ten seconds:

yes | (sleep 10; head -c 10 >/dev/null)

Run top in another terminal and yes sits at 0% CPU, not 100%. It is not busy. It is asleep inside a write() call, waiting for the pipe to drain.

To see the exact call, attach strace to it:

strace -p "$(pgrep -n yes)" -e trace=write

The last line shown is a write(1, ...) that has started and not returned. Nothing is wrong with the process. The kernel simply has nowhere to put the bytes.

How big is the buffer?

On Linux the default capacity is 65536 bytes (64 KiB, sixteen pages). You can ask a pipe directly:

import fcntl, os

r, w = os.pipe()
print(fcntl.fcntl(w, 1032))  # F_GETPIPE_SZ, prints 65536 by default

So the writer can get roughly 64 KiB ahead of the reader and no further. Once that much is queued, the next write blocks. As soon as the reader consumes some data, the kernel wakes the writer and it carries on.

Quick detour: why the blocking is a feature

Hang on, why not just grow the buffer without limit? Because the buffer is kernel memory, and an unbounded one would let a runaway yes eat the machine. A small bound turns a memory problem into a scheduling problem: the fast side simply waits.

This is backpressure with no code at all. Every shell pipeline you have ever written is rate-limited by its slowest stage, and nobody had to design a flow-control protocol. It is rather elegant, honestly.

What the writer actually sees

The behaviour depends on how the write end is opened:

  • Blocking (the default): write() sleeps until there is room, then completes.
  • Non-blocking (O_NONBLOCK): a full pipe makes write() fail with EAGAIN, or return a short count if some bytes fit.
  • Reader gone: the write fails with EPIPE and the process gets SIGPIPE, which is the other half of this topic.

The short count matters. For a blocking write larger than the free space, the kernel may block partway through, but writes of up to PIPE_BUF bytes (4096 on Linux) are guaranteed atomic and will not be interleaved with other writers.

Things that make it worse

The usual culprit is a reader that is slow for a reason you did not expect. A few real ones:

  • The reader does synchronous work per line, such as a database insert or a network call.
  • The reader is a pager or terminal that is itself waiting for a human.
  • Two stages deadlock: stage A waits on B's output while B waits on A, and both pipes are full.
  • The reader forgot to drain a second pipe, typically stderr, in a program that spawns children.

That last one bites people writing process wrappers. If you capture a child's stdout and stderr and read only one, the other fills to 64 KiB and the child freezes mid-run. Read both concurrently, or redirect one to a file.

Changing the size, and whether you should

You can enlarge a pipe with F_SETPIPE_SZ, up to /proc/sys/fs/pipe-max-size (1 MiB by default for unprivileged processes):

import fcntl, os

r, w = os.pipe()
fcntl.fcntl(w, 1031, 1 << 20)  # F_SETPIPE_SZ, ask for 1 MiB
print(fcntl.fcntl(w, 1032))

A bigger buffer smooths out bursts. It does not fix a reader that is permanently slower than the writer; it just delays the stall. If you need real decoupling, put an explicit queue or a file between the stages, or use a tool like pv or mbuffer to absorb the difference and show you throughput while it does.

The constant values above are the Linux ones; other Unixes differ in default capacity, so check before relying on 64 KiB elsewhere.

Diagnosing a stall quickly

  1. Find the stuck writer with ps or top; it will show low CPU and state S.
  2. Check /proc/<pid>/stack or run strace -p to confirm it is parked in write.
  3. Look at the process on the read end of that pipe. That is the one to fix, not the writer.

The writer is almost never the problem. It is doing exactly what the kernel told it to do: wait its turn.

← All articles