Articles / Go

1097 words

  • go
  • net-http
  • context
  • basecontext
  • conncontext
  • middleware

Per-Connection Values in Go with BaseContext and ConnContext

Your handler knows plenty about the request and almost nothing about the connection it arrived on. It gets a *http.Request with URL, headers, remote address and TLS state. What it does not get is anything connection-level you have decided matters:

  • Which listener accepted it?
  • Is this the first request on that connection or the four hundredth?
  • Which connection ID goes in the log line so it lines up with the proxy's log?

Two fields on http.Server exist for exactly this, and hardly anyone uses them: BaseContext and ConnContext. Both have been there since Go 1.13. I tested everything below on Go 1.22.

Three layers of context, and who creates each one

BaseContext is called once per listener. Its signature is func(net.Listener) context.Context, and it runs when you call Serve (or ListenAndServe, which calls it for you). Whatever it returns becomes the root of every context on that listener.

Two edge cases: leave it nil and you get context.Background(); return nil from it and the server panics.

ConnContext is called once per accepted connection. Its signature is func(ctx context.Context, c net.Conn) context.Context. It receives a context derived from the base context, and what you return is the parent of every request context on that connection.

So the hierarchy runs listener, then connection, then request:

  1. Base context, per listener.
  2. Connection context, per connection.
  3. Request context, per request.

Values set at the top are visible all the way down through r.Context().

Tagging connections with an ID and a request counter

Here is a small server that tags each listener by address, gives each connection an ID, and counts requests on it:

package main

import (
	"context"
	"fmt"
	"log"
	"net"
	"net/http"
	"sync/atomic"
)

type ctxKey int

const (
	listenerKey ctxKey = iota
	connKey
)

type connInfo struct {
	id       uint64
	remote   string
	requests atomic.Int64
}

func newServer(h http.Handler) *http.Server {
	var nextID atomic.Uint64
	return &http.Server{
		Handler: h,
		BaseContext: func(l net.Listener) context.Context {
			return context.WithValue(context.Background(), listenerKey, l.Addr().String())
		},
		ConnContext: func(ctx context.Context, c net.Conn) context.Context {
			ci := &connInfo{id: nextID.Add(1), remote: c.RemoteAddr().String()}
			return context.WithValue(ctx, connKey, ci)
		},
	}
}

func handler(w http.ResponseWriter, r *http.Request) {
	ci := r.Context().Value(connKey).(*connInfo)
	n := ci.requests.Add(1)
	listener := r.Context().Value(listenerKey)
	fmt.Fprintf(w, "listener=%v conn=%d request=%d\n", listener, ci.id, n)
}

func main() {
	srv := newServer(http.HandlerFunc(handler))
	srv.Addr = "127.0.0.1:8080"
	log.Fatal(srv.ListenAndServe())
}

Hit it with curl -s localhost:8080 localhost:8080 and curl reuses the connection, so you see conn=1 request=1 then conn=1 request=2. Run curl again and you get conn=2.

Two details worth copying:

  • The context keys are an unexported type, not bare strings, so nothing else in the process can collide with them.
  • The per-connection value is a pointer to a struct, not a snapshot, so handlers can read and update shared connection state.

That second point has a catch, which is next.

HTTP/2 makes "per connection" concurrent

On HTTP/1.1 a connection handles one request at a time, so a plain int counter in connInfo would happen to work. HTTP/2 changes that. One connection carries many concurrent streams, each becomes a request in its own goroutine, and they all hold the same *connInfo. A plain field is then a data race.

That is why the struct above uses atomic.Int64. Anything you mutate on a per-connection value needs an atomic or a mutex, even if your test client only speaks HTTP/1.1.

The connection values do propagate over HTTP/2. I ran three sequential requests against a TLS test server with HTTP/2 enabled, and all three reported the same connection ID, with the listener value from BaseContext intact.

ConnContext runs in the accept loop, so keep it boring

This is the bit I did not expect. The server calls ConnContext synchronously in the loop that calls Accept, before it starts the goroutine that serves the connection. So a slow ConnContext delays every connection behind it.

I checked by sleeping for one second inside ConnContext and firing three concurrent requests at a plain HTTP server. They completed at one, two and three seconds: strictly serialised. Any of these in there is a self-inflicted stall on the whole listener:

  • a DNS lookup
  • a database call
  • a lock that someone else holds

The TLS handshake has not happened yet

Quick detour, because this one follows from the same fact. Under ServeTLS, the net.Conn you are handed is a *tls.Conn (my test printed exactly that), but the handshake has not run yet. It happens later, in the per-connection goroutine.

So you cannot read the client certificate or negotiated protocol in ConnContext. Nor should you call Handshake() yourself to get them: that puts a network round trip with an untrusted peer inside the accept loop. Do that sort of thing lazily, in the handler, from r.TLS.

What it is good for

ConnContext suits cheap, local work:

  • allocating an ID
  • copying the remote address
  • grabbing a value off the conn's own type

That last one is handy with listener wrappers. If you use one (for example one that parses a PROXY protocol header), the connection you receive is your wrapper type. A type assertion in ConnContext is then a tidy way to pass the real client address down to handlers.

It will not tell you when the connection closes

It is tempting to think a connection-scoped context gives you a "connection closed" callback. It does not, at least not through ConnContext. The server derives its own cancellable context from whatever you return, and request contexts are cancelled when the client goes away. But the context you return is never cancelled by the server on your behalf.

If you need to know that connection 7 has finished so you can free something, that is what ConnState and StateClosed are for. The two pair up well: ConnContext creates the value, ConnState disposes of it, and the connection ID is the link if you keep them in a map.

Also be wary of stashing the net.Conn itself in the context so handlers can poke at it. The server owns that connection and is reading from it.

  • Pulling out addresses, or calling SyscallConn for read-only socket options, is fine.
  • Reading or writing bytes from a handler is a bug.
  • For anything that really takes the connection over, there is Hijack.

BaseContext doubles as a kill switch

Server.Shutdown waits politely, and that is sometimes the problem. It stops accepting and waits for in-flight handlers, but it does not cancel their contexts. If a handler is stuck on a slow upstream and honours its context, you have nothing to signal it with. Make the base context yours:

baseCtx, cancelAll := context.WithCancel(context.Background())
defer cancelAll()

srv := &http.Server{
	Handler:     mux,
	BaseContext: func(net.Listener) context.Context { return baseCtx },
}

// ... on SIGTERM:
shutdownCtx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
defer cancel()
if err := srv.Shutdown(shutdownCtx); err != nil {
	cancelAll() // grace period expired: tell the stragglers to give up
	srv.Close()
}

Every request context descends from baseCtx, so cancelAll() reaches all of them at once. Handlers that ignore their context will not care, of course; this only helps code that was already written to respect cancellation.

One more use, since the listener is passed in. If one server serves both a public and an admin listener, BaseContext is where you tag which is which, keyed on l.Addr(). A handler can then refuse admin routes unless the request came in on the admin listener, which is a sturdier check than looking at the Host header a client chose for itself.

← All articles