Articles / Go

776 words

  • go
  • slices
  • append
  • aliasing
  • memory
  • debugging

Why a Go Slice Append Sometimes Changes Your Other Slice

You append to one slice and a completely different slice changes underneath you. No race, no pointer in sight. This is ordinary Go behaviour, and it bites because the same code is fine on Tuesday and wrong on Wednesday, depending on a capacity you never looked at.

Here is the smallest reproduction:

package main

import "fmt"

func main() {
	a := []int{1, 2, 3, 4, 5}
	b := a[:2]
	b = append(b, 99)

	fmt.Println(a) // [1 2 99 4 5]
	fmt.Println(b) // [1 2 99]
}

We never touched a, yet a[2] is now 99.

A slice is a window, not a container

A slice value is three words: a pointer to an array, a length and a capacity. Slicing copies the header, not the data. So b := a[:2] gives you a new header with length 2, pointing at the same array as a, and with capacity 5 (everything from the start of b to the end of the array).

That capacity is the trap. append looks at it and asks one question:

  • Is there room (len < cap)? Write the new element into the existing array, at the index just past the length, and return a header with a longer length.
  • No room? Allocate a bigger array, copy everything across, write the element there, and return a header pointing at the new array.

In our example b has length 2 and capacity 5, so there is room. The 99 lands in slot 2 of the shared array, which is exactly where a[2] lives.

The same code, with a different outcome

Now append to the full-length slice instead:

a := []int{1, 2, 3, 4, 5}
c := append(a, 6) // len 5, cap 5: no room, so reallocate
c[0] = 100

fmt.Println(a[0]) // 1
fmt.Println(c[0]) // 100

Here c got a fresh array, so the two slices have gone their separate ways. Whether they alias depends on spare capacity, and spare capacity depends on how the slice was built, which may be several function calls away from where you append.

Quick detour: why append returns a value

This also explains why you must write b = append(b, x) rather than just calling append(b, x). The header is passed by value. If append reallocated, the only place the new pointer exists is in the returned header. If it did not, the old header still has its old length and cannot see the new element. Either way, the return value is the real result.

Where this really hurts: forking from a shared prefix

The nasty version is when you branch from a common base, as you do in recursive search, path building or permutations:

base := make([]string, 0, 4)
base = append(base, "a", "b")

x := append(base, "x")
y := append(base, "y")

fmt.Println(x) // [a b y]
fmt.Println(y) // [a b y]

Both appends found spare capacity in base, so both wrote into slot 2 of the same array. x and y are two headers over one array, and the second write won.

It gets worse: remove the spare capacity (say make([]string, 0, 2)) and the output becomes [a b x] [a b y], which is what you wanted. A bug that vanishes when you change an unrelated allocation size is a lovely thing to find in code review at 5pm on a Friday.

Three ways to stop it

Pick based on how much you care about the extra allocation.

  1. Cap the capacity with a full slice expression. base[:len(base):len(base)] gives a slice whose capacity equals its length, so append must reallocate.
  2. Use slices.Clip, which does exactly that in a readable way.
  3. Copy up front with slices.Clone when you want an independent slice from the start.
import "slices"

x := append(slices.Clip(base), "x")
y := append(slices.Clip(base), "y")
// x is [a b x], y is [a b y], base is untouched

The three-index form s[low:high:max] is worth remembering on its own. The third number sets the capacity limit, so you can hand a function a sub-slice it can read and even append to, without it being able to scribble over the rest of your array.

Quick detour: can you predict when it reallocates?

Not reliably, and you should not try. The growth rule is an implementation detail that has changed between Go releases (it also rounds up to allocator size classes), so the capacity after an append is not something the language promises. Read cap() if you need it now, and design code so correctness does not depend on the answer.

A rule that avoids most of it

Treat any slice you did not create yourself as read-only unless the contract says otherwise. If a function takes a slice and appends to it, either document that it may clobber the caller's backing array, or have it work on a clone. When you return a sub-slice of an internal buffer, clip it first.

The same aliasing also keeps a whole large array alive when you hold a tiny sub-slice of it, which is a memory problem rather than a correctness one, and has the same cure: copy what you need.

If you suspect this bug, print len and cap of both slices side by side, or compare &a[0] with &b[0]. Same address, shared array.

← All articles