Articles / Go

970 words

  • go
  • errgroup
  • concurrency
  • goroutines
  • context
  • sync-waitgroup

Cancel a Group of Goroutines Without Hand-Rolled WaitGroups

Fire off N requests across goroutines, one fails, and the other N-1 keep running anyway because nothing told them to stop. You get the right error back eventually, several seconds late, after every other goroutine has finished doing work nobody wanted any more.

This bug turns up in almost every Go codebase that fans work out. The usual first fix bolts together:

  • a sync.WaitGroup
  • a mutex-protected "first error" variable
  • a context.CancelFunc called from inside the failing goroutine

It works, but you end up writing the same twenty lines in every package that does fan-out work. golang.org/x/sync/errgroup is that twenty lines, written once, by people who thought about the edge cases you haven't hit yet.

The boilerplate version

Here's roughly what people write before they discover errgroup exists: fetch a batch of URLs concurrently, stop everything as soon as one fails, return the first error.

func fetchAll(ctx context.Context, urls []string) ([][]byte, error) {
	ctx, cancel := context.WithCancel(ctx)
	defer cancel()

	var (
		wg       sync.WaitGroup
		mu       sync.Mutex
		firstErr error
	)
	results := make([][]byte, len(urls))

	for i, url := range urls {
		wg.Add(1)
		go func(i int, url string) {
			defer wg.Done()
			body, err := fetch(ctx, url)
			if err != nil {
				mu.Lock()
				if firstErr == nil {
					firstErr = err
					cancel()
				}
				mu.Unlock()
				return
			}
			results[i] = body
		}(i, url)
	}
	wg.Wait()
	if firstErr != nil {
		return nil, firstErr
	}
	return results, nil
}

None of this is wrong, exactly. It's just a lot of ceremony for "run these concurrently, stop on first failure, tell me what went wrong".

Every field in that struct literal (wg, mu, firstErr) exists purely to compensate for one thing: a plain WaitGroup has no concept of failure. It just counts goroutines down to zero and has nothing to say about whether any of them were unhappy on the way.

The errgroup version drops the mutex and the cancel plumbing

errgroup.Group is that same pattern, packaged. errgroup.WithContext gives you a group and a derived context that gets cancelled the moment any goroutine in the group returns a non-nil error, or as soon as Wait returns, whichever happens first.

import "golang.org/x/sync/errgroup"

func fetchAll(ctx context.Context, urls []string) ([][]byte, error) {
	g, ctx := errgroup.WithContext(ctx)
	results := make([][]byte, len(urls))

	for i, url := range urls {
		g.Go(func() error {
			body, err := fetch(ctx, url)
			if err != nil {
				return err
			}
			results[i] = body
			return nil
		})
	}

	if err := g.Wait(); err != nil {
		return nil, err
	}
	return results, nil
}

That's the whole thing. No mutex, no manual cancel plumbing, no separate error variable. Here's what happens:

  1. g.Go starts a goroutine.
  2. If it returns an error, errgroup records the first one and cancels the derived ctx. Every other in-flight fetch call is presumably already respecting that, because it was passed the context and checks it in its HTTP calls.
  3. Wait blocks until every goroutine in the group has returned, then hands back that first error (or nil).

Quick detour if you're on anything before Go 1.22: the loop variable capture in that example is fine only because Go 1.22 changed loop variables to be per-iteration. On older toolchains you'd need i, url := i, url inside the loop body before the closure, same as with any goroutine-per-iteration pattern.

SetLimit bounds concurrency without a semaphore channel

Firing off a goroutine per URL is fine for a handful of requests. For a few thousand it's a good way to open a few thousand sockets at once and get rate-limited, or worse.

The classic fix is a buffered channel used as a semaphore. errgroup has this built in via SetLimit:

g, ctx := errgroup.WithContext(ctx)
g.SetLimit(8)

for _, url := range urls {
	url := url
	g.Go(func() error {
		return fetchInto(ctx, url, store)
	})
}
if err := g.Wait(); err != nil {
	return err
}

With a limit set, g.Go blocks the caller until a slot is free, so at most 8 fetches run at once. A few related details:

  • TryGo does the same check but returns false immediately instead of blocking if the group is already at capacity. Useful when you'd rather skip or defer work than queue for a goroutine slot.
  • Calling SetLimit with a negative number removes the limit again.
  • The zero value has no limit at all.

The gotchas that actually bite

A few things about errgroup are easy to assume incorrectly, usually because the API looks so much like a tidier WaitGroup that you stop thinking about what's different underneath.

1. It does not catch panics

If a function passed to Go panics, that panic unwinds its goroutine exactly as it would for any other goroutine. In Go that means it crashes the whole process, not just the caller of Wait.

This isn't errgroup being careless; it's the same rule as everywhere else. A panic in one goroutine can't be recovered by a different goroutine. If a piece of work might panic and you want an error instead of a process death, wrap it in your own recover inside the function you pass to Go.

2. Wait only ever returns the first error

If three of your ten goroutines fail, you get told about one of them, not three. The other two errors are gone, discarded, never logged unless the individual goroutines log them themselves before returning.

If you actually need to see every failure, log inside each goroutine before returning the error, or collect them into a slice behind a mutex yourself. Errgroup picks a winner and cancels the rest; it doesn't aggregate.

3. Cancellation is still cooperative

It's still just context.Context underneath. Calling the derived context's cancellation doesn't reach into a running goroutine and stop it; it only sets a channel that the goroutine has to be checking.

A fetchInto that ignores ctx entirely, or that's blocked on something that doesn't watch ctx.Done(), will run to completion regardless of what the rest of the group is doing.

Errgroup buys you the plumbing to signal cancellation cleanly. It doesn't buy you goroutines that actually stop when asked. That part's still on whatever code you write inside Go.

4. You don't always need WithContext

Easy to miss: the zero-value errgroup.Group works fine without WithContext at all. It just collects the first error and doesn't cancel anything.

That's the right shape when the goroutines genuinely don't depend on each other and there's nothing useful to cancel. No point wiring up a context just to satisfy a habit.

It's in x/sync, and that's fine

errgroup lives in golang.org/x/sync, not the standard library proper, but it's maintained by the Go team under the same compatibility promise as the rest of x/.

It's about as close to "should have been in sync" as a dependency gets. Reaching for it beats hand-rolling the WaitGroup-plus-mutex-plus-cancel pattern for the tenth time.

← All articles