Phone:

Hidden from the page source until you click: friction against scrapers, not a guarantee.

Email:

[email protected]

Category:

Go

Tags:
  • go
  • net-http
  • http-request
  • roundtripper
  • middleware
  • debugging

Go's http.Request.Clone: Which Fields Get Copied, Which Are Shared

There is a bug I have seen in a few Go codebases, and it looks like this: an http.RoundTripper wrapper adds an Authorization header, the caller's original request quietly gains that header too, and somebody logs it. Or two goroutines fan out one request with different tracing headers and get a concurrent map write panic. The cause is nearly always the same: a "copy" of a request that was not really a copy.

The slightly awkward part is that the title of this post could mislead you in either direction. Request.Clone does not share your headers. Request.WithContext does. They look like siblings, and they are not.

Start with the experiment

package main

import (
	"context"
	"fmt"
	"io"
	"net/http"
	"strings"
)

func main() {
	r, err := http.NewRequest("POST", "https://example.com/", strings.NewReader("payload"))
	if err != nil {
		panic(err)
	}
	r.Header.Set("X-Trace", "original")

	// WithContext: shallow copy of the struct.
	a := r.WithContext(context.Background())
	a.Header.Set("X-Trace", "via-WithContext")
	fmt.Println(r.Header.Get("X-Trace")) // via-WithContext

	// Clone: Header is deep-copied.
	b := r.Clone(context.Background())
	b.Header.Set("X-Trace", "via-Clone")
	fmt.Println(r.Header.Get("X-Trace")) // still via-WithContext

	// ...but the Body is the very same reader.
	got, _ := io.ReadAll(b.Body)
	fmt.Println(string(got)) // payload
	rest, _ := io.ReadAll(r.Body)
	fmt.Printf("%q\n", rest) // ""
}

Three different behaviours in one small program. The WithContext copy shares the Header map with the original, so a write through one is visible through the other. The Clone copy gets its own map. And both share the body, so reading it from the clone drains it for the original as well.

What Clone actually copies

The documentation says Clone returns a deep copy of the request with its context changed to the one you pass. In the standard library implementation, that means it copies the struct and then replaces these with fresh copies: URL, Header, Trailer, TransferEncoding, Form, PostForm and MultipartForm. Those are the fields where a caller would plausibly mutate something in place.

Everything else is copied by value, which for pointers, interfaces and funcs means the copy points at the same thing. The ones that matter in practice:

  • Body: an io.ReadCloser, so one underlying stream, one read position.
  • GetBody: the func is shared, but each call to it returns a fresh reader, which is what you want.
  • TLS: a pointer to the connection state on a server request.
  • Response: the redirect-related pointer, if set.

So "deep copy" in the docs is a slight overstatement, and I would read it as "deep copy of the bits you would normally mutate, minus the body". Actually, this bit is where the surprise lives. Nothing in the name tells you the body is excluded, and you cannot deep-copy a stream without reading it, so it could not sensibly be otherwise.

WithContext is the shallow one

WithContext is documented as returning a shallow copy of the request with its context changed. That is exactly what you saw above. It is cheap (one struct allocation), and that is the reason it exists: it is what you use when you only want to attach a deadline or a value and are not going to touch anything else.

A little history, because older advice contradicts itself here. Earlier releases of WithContext also deep-copied the URL; that was dropped later, so it is now a plain struct copy. If you are reading a blog post from years ago that says "WithContext copies the URL", check it against the Go version you actually use. The header behaviour, in any case, has been shared all along.

Where this bites: RoundTripper wrappers

The RoundTripper contract says a RoundTrip implementation must not modify the request, other than consuming and closing the body. That rule is easy to break by accident, because the tempting code is two lines:

// Broken: mutates the caller's request.
func (t *authTransport) RoundTrip(req *http.Request) (*http.Response, error) {
	req.Header.Set("Authorization", "Bearer "+t.token)
	return t.base.RoundTrip(req)
}

The caller's req.Header now carries your token. If they retry, log the request, or hand it to a second client for a different host, the token goes along with it. The fix is one Clone:

func (t *authTransport) RoundTrip(req *http.Request) (*http.Response, error) {
	r2 := req.Clone(req.Context())
	r2.Header.Set("Authorization", "Bearer "+t.token)
	return t.base.RoundTrip(r2)
}

Note that Clone takes a context and panics if you pass nil, so req.Context() is the natural argument when you want to keep the existing one. The body is still shared here, which is fine: the transport is going to consume it once, exactly as the contract expects.

Where this bites: retries

Retry loops are the second trap, and this is where the shared body hurts. Cloning a request per attempt gives you clean headers, but attempt two would send an already-drained body. Use GetBody, which http.NewRequest fills in automatically for *bytes.Buffer, *bytes.Reader and *strings.Reader bodies:

func attempt(c *http.Client, req *http.Request) (*http.Response, error) {
	r := req.Clone(req.Context())
	if req.GetBody != nil {
		body, err := req.GetBody()
		if err != nil {
			return nil, err
		}
		r.Body = body
	}
	r.Header.Set("X-Attempt-Id", newID()) // safe: r has its own Header
	return c.Do(r)
}

(newID is whatever you use to generate an identifier; it is not part of the standard library.) If GetBody is nil and Body is non-nil, the body came from an arbitrary reader and you cannot replay it. Do not retry those requests blindly; buffer the payload up front if you need retries, or fail the retry.

Concurrency: the panic version

Go maps panic on concurrent writes, which is a fatal error rather than a recoverable one. Fan out a single request to several goroutines using WithContext, have each set its own header, and you will eventually see "concurrent map writes" with a stack pointing into Header.Set. Nothing looks wrong at the call site, since each goroutine had "its own" request. With Clone each has a real private header map, and go test -race stays quiet.

Cloning also costs something: it allocates a new header map and the URL, and for big multipart forms it does real work. For a hot path where you only need a different context, WithContext is the right tool, as long as you promise not to write to anything on the result. A rule I use: if the code after the call contains a .Set, .Add or .Del, or assigns to URL fields, it should be Clone.

One more shared thing: Header values you stored

Clone copies the header map and the slices of values, so appending to b.Header["X"] cannot leak back. But if you later stash a pointer or a slice in a request context value and mutate that, the clone shares it, because contexts are immutable chains, not copies. That is the same trap in a different place, and it is not something Clone can save you from.

← Back to the blog