Go's http.Transport Idle Pool: Why MaxIdleConnsPerHost Is 2

Here is a symptom I find oddly satisfying to chase. A service calls one upstream API, and under bursty load its latency has a strange floor: every burst pays for fresh TCP and TLS handshakes, even though you are sure you are reusing connections. You have a shared http.Client, you drain and close the bodies, and netstat still shows a heap of sockets in TIME_WAIT.

The culprit is usually one number: DefaultMaxIdleConnsPerHost, which is 2.

What the knobs actually do

http.DefaultTransport sets MaxIdleConns to 100 and IdleConnTimeout to 90 seconds. That 100 sounds generous. But it is a total across all hosts. The per-host limit, MaxIdleConnsPerHost, is separate, and when it is zero the transport uses DefaultMaxIdleConnsPerHost, a constant of 2.

Note the word "idle". Neither setting limits how many connections you can have open at once; that is MaxConnsPerHost, which is zero (unlimited) by default. So Go will happily open 50 connections to one host for 50 concurrent requests. The trouble comes when they finish: the transport returns each connection to the idle pool, sees that the host already has 2 idle, and closes the rest. Forty-eight sockets are torn down, and the next burst of 50 has to dial 48 new ones.

Why 2, though?

As far as I can tell, it is historical. The old HTTP/1.1 specification (RFC 2616) said clients should not keep more than two connections to a server, and browsers and libraries of that era followed suit. Go's default reflects that lineage rather than any measurement of modern API traffic. I would treat that as my reading of the history, not something the docs spell out. What the docs do say is that the default is 2, and that is enough to plan around.

For a client that talks to many different hosts one at a time, 2 is fine. For a service that fans out to one backend (a payment API, an internal service, an object store), it is far too low.

Measuring it instead of guessing

Rather than trusting netstat, count new connections on the server side. This test server holds each request for 50ms so the requests genuinely overlap, and counts StateNew events. It fires two bursts of 20 requests with a pause between, so the first burst's connections have all been returned to the pool before the second starts.

package main

import (
	"fmt"
	"io"
	"net/http"
	"net/http/httptest"
	"sync"
	"sync/atomic"
	"time"
)

func burst(c *http.Client, url string, n int) {
	var wg sync.WaitGroup
	for range n {
		wg.Add(1)
		go func() {
			defer wg.Done()
			resp, err := c.Get(url)
			if err != nil {
				return
			}
			defer resp.Body.Close()
			io.Copy(io.Discard, resp.Body)
		}()
	}
	wg.Wait()
}

func run(name string, perHost int) {
	var dials atomic.Int64
	srv := httptest.NewUnstartedServer(http.HandlerFunc(
		func(w http.ResponseWriter, r *http.Request) {
			time.Sleep(50 * time.Millisecond)
			fmt.Fprint(w, "ok")
		}))
	srv.Config.ConnState = func(_ net.Conn, s http.ConnState) {
		if s == http.StateNew {
			dials.Add(1)
		}
	}
	srv.Start()
	defer srv.Close()

	tr := http.DefaultTransport.(*http.Transport).Clone()
	tr.MaxIdleConnsPerHost = perHost
	c := &http.Client{Transport: tr, Timeout: 10 * time.Second}

	burst(c, srv.URL, 20)
	first := dials.Load()
	time.Sleep(100 * time.Millisecond)
	burst(c, srv.URL, 20)
	fmt.Printf("%s: first burst %d new conns, second burst %d new conns\n",
		name, first, dials.Load()-first)
}

func main() {
	run("default (2)", 0)
	run("raised (20)", 20)
}

You will need "net" in the imports for the ConnState callback; I left it out above to keep the shape readable, but it will not compile without it. Timing makes the exact numbers wobble, but the pattern is stable: the first burst dials roughly 20 either way, the default client dials roughly 18 more on the second burst, and the raised client dials close to none.

The fix, and the way to write it

Clone the default transport and change the one field. Do not build a bare &http.Transport{} unless you mean to, because you will silently lose the proxy-from-environment setting, the dialer timeouts, and the HTTP/2 attempt that DefaultTransport gives you.

tr := http.DefaultTransport.(*http.Transport).Clone()
tr.MaxIdleConnsPerHost = 32
tr.MaxIdleConns = 0 // 0 means no overall limit
client := &http.Client{Transport: tr, Timeout: 15 * time.Second}

Setting MaxIdleConns to zero removes the global cap. If you have raised the per-host number and left the global one at 100, fine, but it is worth checking the two agree with how many hosts you actually call. Pick the per-host value from your real peak concurrency to that host, not from a round number. If the peak is 200 and you set 32, you still throw 168 away after each burst.

Checking reuse from the client side

You do not need a test server in production. net/http/httptrace tells you per request whether the connection was reused:

trace := &httptrace.ClientTrace{
	GotConn: func(i httptrace.GotConnInfo) {
		log.Printf("reused=%v idle=%v idleTime=%s", i.Reused, i.WasIdle, i.IdleTime)
	},
}
req = req.WithContext(httptrace.WithClientTrace(req.Context(), trace))

Log a sample of these and the reuse ratio is obvious. A ratio that drops right after each burst is this exact problem.

Two caveats worth knowing

First, with HTTP/2 most of this stops mattering. A single connection multiplexes many streams, so the pool holds one connection per host and the per-host idle limit is barely exercised. If your upstream speaks HTTP/2 over TLS you may never see the symptom. It bites hardest with plain HTTP/1.1: internal services over cleartext, or TLS servers that do not negotiate h2.

Second, a bigger idle pool is not free. Idle connections hold file descriptors on both ends and can be closed by the server or a middlebox at any moment; IdleConnTimeout (90 seconds by default) is what stops you keeping ones the far side has already dropped. If the upstream has a shorter idle timeout than yours, the transport may pick a connection that is half-closed, and you will see the occasional retried or failed request. Keep your timeout a little under theirs.

The two-connection default is a sensible number for 1990s browsers and a bad one for a backend service. Changing it is one line; knowing you need to is the actual work.