Go's io.Pipe Has No Buffer: How a Stalled Reader Freezes Writes

This program never prints anything:

package main

import (
	"fmt"
	"io"
)

func main() {
	r, w := io.Pipe()
	defer r.Close()
	defer w.Close()

	if _, err := w.Write([]byte("hello")); err != nil {
		panic(err)
	}

	data, err := io.ReadAll(r)
	if err != nil {
		panic(err)
	}
	fmt.Println(string(data))
}

The write does not place hello somewhere for a later read. It waits for a reader to take all five bytes. The reader is below the write in the same goroutine, so execution never reaches it. Both halves exist, all the methods are valid, and the program is nevertheless stuck.

This is not an unfortunate edge case. It is the defining property of io.Pipe: it is a synchronous in-memory pipe with no internal buffer.

A Write is a rendezvous, not a deposit

A helpful mental model is an unbuffered channel carrying byte slices, although the implementation has some extra machinery to handle partial reads, closure and serialised calls. A call to Write(p) completes only after one or more calls to Read have consumed all of p.

The sizes need not match. If the writer offers 32 KiB and the reader uses a 4 KiB buffer, the writer remains blocked across eight reads. It does not return after the first 4 KiB has moved. Conversely, a large read may receive the contents of one smaller write and return immediately; Read is not obliged to fill its buffer.

Actually, this bit is useful when debugging: a goroutine stuck in (*PipeWriter).Write does not necessarily mean nobody has read anything. The consumer might have read most of that particular write and then stalled on its final few bytes.

The zero-buffer design gives io.Pipe natural backpressure. A fast producer cannot quietly accumulate an arbitrary amount of data behind a slow consumer. Memory use stays predictable, and the producer runs at roughly the consumer's pace. That is excellent for streaming a generated archive, compressed output or encoded data. It also means the consumer controls the producer's ability to make progress.

The goroutine fixes the first deadlock

The smallest correction is to run one side concurrently. The producer must also close its end, otherwise io.ReadAll waits forever for an EOF which never arrives:

package main

import (
	"fmt"
	"io"
)

func main() {
	r, w := io.Pipe()

	go func() {
		_, err := w.Write([]byte("hello"))
		_ = w.CloseWithError(err)
	}()

	data, err := io.ReadAll(r)
	if err != nil {
		panic(err)
	}
	fmt.Println(string(data))
}

CloseWithError(nil) behaves like a normal writer close, so the reader sees EOF after the bytes. If producing the stream fails, the supplied error crosses the pipe and is returned by the reader instead. This is usually more useful than separately logging an error in a goroutine and leaving the consumer wondering why its input ended early.

There is another ordering trap. Starting the producer in a goroutine is not enough if the main goroutine waits for that producer before it starts consuming:

done := make(chan error, 1)

go func() {
	_, err := w.Write(payload)
	done <- err
}()

// Deadlock: Write awaits a reader, while this receive awaits Write.
if err := <-done; err != nil {
	return err
}

_, err := io.Copy(destination, r)
return err

The stream must be consumed before awaiting the producer, or consumption and waiting must be coordinated so that either failure closes the other end.

The nastier freeze happens when a reader gives up

Suppose the consumer parses a header, decides the input is invalid, and returns without closing the PipeReader. The producer may already be inside Write. Since no further read will consume the remaining bytes, that goroutine stays blocked indefinitely.

Garbage collection does not rescue it. A reachable blocked goroutine keeps the pipe state alive. Nor does cancelling an unrelated context.Context magically interrupt io.Pipe; neither PipeReader nor PipeWriter has deadline or context methods.

Closing the opposite end is the interruption mechanism. Closing the reader makes a blocked or subsequent write return io.ErrClosedPipe. Using CloseWithError lets it return a more informative error.

A reusable pipeline can make that ownership explicit:

package pipeline

import (
	"errors"
	"io"
)

func Run(
	produce func(io.Writer) error,
	consume func(io.Reader) error,
) error {
	r, w := io.Pipe()
	producerDone := make(chan error, 1)

	go func() {
		err := produce(w)
		_ = w.CloseWithError(err)
		producerDone <- err
	}()

	consumeErr := consume(r)
	_ = r.CloseWithError(consumeErr)

	produceErr := <-producerDone
	return errors.Join(consumeErr, produceErr)
}

Several small details matter here. The result channel is buffered, so the producer can report its result even if the consumer is still unwinding. The consumer closes the reader on every return path, which releases a producer blocked in Write. Only then does the function wait for the producer. Finally, errors.Join preserves failures from both sides.

This function deliberately treats a consumer which stops early as ending the whole pipeline. If early termination is normal, such as reading just one record, define that protocol explicitly. You might translate io.ErrClosedPipe into success, but only when the consumer's successful early return genuinely means the remaining output is unwanted. Globally ignoring io.ErrClosedPipe is a splendid way to hide an accidental truncation.

Context cancellation still needs a bridge

If cancellation must interrupt a blocked pipe operation, arrange for cancellation to close an end:

go func() {
	<-ctx.Done()
	_ = r.CloseWithError(ctx.Err())
}()

That is the core idea, but lifecycle matters. If the operation finishes normally, a permanent watcher goroutine remains until the context is eventually cancelled. In library code, give the watcher a completion channel or derive a child context and cancel it when the pipeline ends. Also remember that closing the pipe only releases pipe operations. It cannot interrupt a producer blocked while reading some other source unless that source supports cancellation or closure too.

Adding bufio does not make the pipe asynchronous

Wrapping the writer with bufio.NewWriter can move the stall, but cannot remove it. Small writes succeed until the buffer fills or somebody calls Flush; then the flush blocks until the reader consumes the data. This may be useful for combining tiny writes, but it is not a general decoupling queue.

If the producer must run ahead, choose storage with an intentional capacity and overflow policy. A bytes.Buffer works when the complete result can be generated before consumption and fits comfortably in memory, though it must not be read and written concurrently without synchronisation. A temporary file suits larger materialised output. A bounded channel of copied byte chunks can provide a controlled amount of read-ahead. An operating-system pipe has a kernel buffer, but that buffer is finite too, so writes eventually block.

io.Pipe is best when backpressure is wanted: two concurrent components, one byte stream, and clear ownership of both closures. If either side may stop, make closing the other side part of the design. Otherwise the apparent streaming optimisation comes with a goroutine quietly parked forever, usually just after the logs claim the interesting work has finished.