Go's http.Hijacker: Where Buffered Bytes Go After an Upgrade

An HTTP client sends an upgrade request followed immediately by the first message in the new protocol. Your Go handler accepts the upgrade, calls Hijack, reads from the returned net.Conn, and waits.

And waits.

The client has definitely sent the message. A packet capture can see it. The kernel has acknowledged it. Yet conn.Read blocks as if the bytes had wandered off for tea.

They have not disappeared. They are probably sitting in a bufio.Reader which net/http filled while parsing the request. Once that happens, the underlying connection has already advanced past them. Reading the raw connection skips over the buffer and waits for newer data.

Hijack returns two views of one connection

The method is slightly deceptive if read too quickly:

Hijack() (net.Conn, *bufio.ReadWriter, error)

The net.Conn is the underlying transport. The bufio.ReadWriter contains the HTTP server's existing reader and a writer attached to the same transport. The Hijacker documentation explicitly warns that the returned reader may contain unprocessed client data.

Imagine the client writes this in one system call:

GET /chat HTTP/1.1
Host: example.test
Connection: Upgrade
Upgrade: chat/1

[4-byte length][first message]

net/http asks its buffered reader for enough data to parse the HTTP request. A network read is not obliged to stop precisely after the second CRLF, so it may collect part or all of the first protocol frame as well.

The ownership now looks like this:

bufio.Reader:  [4-byte length][first message]
net.Conn:      [next bytes not yet read from the socket]

Calling conn.Read starts at the second line. Calling rw.Reader.Read drains the first line and then continues from the connection. That transition is exactly what the buffered reader exists to provide.

This is timing-dependent, which makes the broken version particularly charming. It may work through a slow proxy, fail on localhost, and change again when logging alters the scheduling.

The rule: keep using the returned reader

After hijacking, read the upgraded protocol through rw.Reader, not directly through conn. Do not switch between them. There is no safe way to infer that the buffer is empty forever, because a later buffered read may prefetch again.

On the write side, send the switching response through rw.Writer and flush it before beginning ordinary protocol traffic. A successful write to a buffered writer only means the bytes entered that buffer. Without Flush, the client may wait for the upgrade response while the server waits for the client's first frame. A very tidy deadlock.

RFC 9110's upgrade rules require a server accepting the transition to send a 101 Switching Protocols response containing the selected Upgrade value. Once hijacked, net/http will not construct that response for you.

A small length-prefixed protocol

This handler upgrades an HTTP/1 connection to a toy echo protocol. Each frame begins with a four-byte big-endian length. The format is deliberately dull so the buffering issue remains visible.

package main

import (
	"bufio"
	"encoding/binary"
	"errors"
	"fmt"
	"io"
	"log"
	"net/http"
	"strings"
	"time"
)

const maxFrame = 64 << 10

func headerHasToken(value, wanted string) bool {
	for _, part := range strings.Split(value, ",") {
		if strings.EqualFold(strings.TrimSpace(part), wanted) {
			return true
		}
	}
	return false
}

func handleUpgrade(w http.ResponseWriter, r *http.Request) {
	if r.Method != http.MethodGet ||
		!headerHasToken(r.Header.Get("Connection"), "upgrade") ||
		!headerHasToken(r.Header.Get("Upgrade"), "chat/1") {
		http.Error(w, "upgrade to chat/1 required", http.StatusBadRequest)
		return
	}

	conn, rw, err := http.NewResponseController(w).Hijack()
	if err != nil {
		http.Error(w, "connection cannot be hijacked", http.StatusInternalServerError)
		return
	}
	defer conn.Close()

	if err := conn.SetDeadline(time.Now().Add(2 * time.Minute)); err != nil {
		log.Printf("setting connection deadline: %v", err)
		return
	}

	_, err = rw.WriteString("HTTP/1.1 101 Switching Protocols\r\n" +
		"Connection: Upgrade\r\n" +
		"Upgrade: chat/1\r\n\r\n")
	if err != nil {
		log.Printf("writing upgrade response: %v", err)
		return
	}
	if err := rw.Flush(); err != nil {
		log.Printf("flushing upgrade response: %v", err)
		return
	}

	for {
		payload, err := readFrame(rw.Reader)
		if errors.Is(err, io.EOF) {
			return
		}
		if err != nil {
			log.Printf("reading frame: %v", err)
			return
		}

		if err := writeFrame(rw.Writer, payload); err != nil {
			log.Printf("writing frame: %v", err)
			return
		}
		if err := rw.Flush(); err != nil {
			log.Printf("flushing frame: %v", err)
			return
		}
	}
}

func readFrame(r io.Reader) ([]byte, error) {
	var size uint32
	if err := binary.Read(r, binary.BigEndian, &size); err != nil {
		return nil, err
	}
	if size > maxFrame {
		return nil, fmt.Errorf("frame is %d bytes; maximum is %d", size, maxFrame)
	}

	payload := make([]byte, size)
	if _, err := io.ReadFull(r, payload); err != nil {
		return nil, err
	}
	return payload, nil
}

func writeFrame(w *bufio.Writer, payload []byte) error {
	if len(payload) > maxFrame {
		return fmt.Errorf("frame is %d bytes; maximum is %d", len(payload), maxFrame)
	}

	var header [4]byte
	binary.BigEndian.PutUint32(header[:], uint32(len(payload)))
	if _, err := w.Write(header[:]); err != nil {
		return err
	}
	_, err := w.Write(payload)
	return err
}

func main() {
	http.HandleFunc("/chat", handleUpgrade)
	log.Fatal(http.ListenAndServe(":8080", nil))
}

The frame limit is not decoration. A peer controls the length field, so allocating before checking it turns four bytes into a convenient memory-exhaustion mechanism.

http.NewResponseController(w).Hijack() also copes with middleware wrappers that expose an Unwrap method. A direct w.(http.Hijacker) assertion is valid, but it unexpectedly fails when an otherwise transparent response recorder or middleware wrapper does not forward the optional interface. The default HTTP/1.x response writer supports hijacking; HTTP/2 deliberately does not.

Things the HTTP server no longer owns

Hijacking is a fairly literal transfer of responsibility. After it succeeds, do not use ResponseWriter or Request.Body. The server implementation marks the connection as hijacked, hands its existing buffered reader back, and stops normal HTTP processing for that connection.

You must close the connection, establish suitable deadlines, bound messages, handle malformed frames and decide how idle peers are evicted. The API documentation permits the returned connection to have existing deadlines, so set the policy your protocol actually needs rather than relying on whichever server settings happened to be present.

Actually, this bit catches production services more often than the byte-buffering bug: Server.Shutdown neither closes nor waits for hijacked connections. Track them yourself and arrange protocol-specific shutdown, perhaps using Server.RegisterOnShutdown to begin that process.

Testing the awkward case

An upgrade test should write the complete HTTP request and the first new-protocol frame together, before reading the server's 101 response. This encourages the server's HTTP reader to fetch beyond the headers. Go's own server tests exercise the same case by placing bytes immediately after the request and verifying that the reader returned by Hijack can recover all of them.

Do not assert that rw.Reader.Buffered() has a particular value. Packet boundaries, socket behaviour and scheduling are not part of the contract. The useful assertion is that the first frame is decoded correctly regardless of whether its bytes were already buffered or still waiting in the socket.

The raw connection is still needed for deadlines, addresses and eventual closure. For protocol reads, though, it is the wrong abstraction. Those apparently missing bytes are sitting one layer above it, patiently waiting for the program to read from the object Hijack returned for precisely that purpose.