Go's singleflight: Stop Cache Misses Stampeding the Database
A cache normally makes a database quieter. The awkward moment is when a popular entry expires and 200 requests discover the empty slot at roughly the same time. Each request performs the same query, each receives the same row, and each writes much the same value back into the cache.
The cache has turned one database read into 200. This is variously called a cache stampede, thundering herd or, during an incident, something less printable.
golang.org/x/sync/singleflight deals with one specific part of this problem: duplicate work that overlaps in time. Give it a key and a function, and it allows only one execution of that function for that key. Concurrent callers wait for, or receive, the same result.
Putting singleflight around a cache fill
The package is outside the standard library, so add it to the module first:
go get golang.org/x/syncA realistic loader needs slightly more care than wrapping the database query in Group.Do. In particular, it should check the cache twice and decide explicitly what request cancellation means.
package users
import (
"context"
"errors"
"fmt"
"time"
"golang.org/x/sync/singleflight"
)
var ErrUnexpectedResult = errors.New("unexpected singleflight result type")
type User struct {
ID int64
Name string
}
type Cache interface {
Get(tenantID string, userID int64) (User, bool)
Set(tenantID string, userID int64, user User, ttl time.Duration)
}
type Repository interface {
UserByID(ctx context.Context, tenantID string, userID int64) (User, error)
}
type Service struct {
cache Cache
repo Repository
loads singleflight.Group
}
func (s *Service) User(
ctx context.Context,
tenantID string,
userID int64,
) (User, error) {
if user, ok := s.cache.Get(tenantID, userID); ok {
return user, nil
}
key := fmt.Sprintf("%d:%s:%d", len(tenantID), tenantID, userID)
resultCh := s.loads.DoChan(key, func() (any, error) {
// Another load may have completed between our first cache check
// and becoming the singleflight leader.
if user, ok := s.cache.Get(tenantID, userID); ok {
return user, nil
}
loadCtx, cancel := context.WithTimeout(
context.WithoutCancel(ctx),
2*time.Second,
)
defer cancel()
user, err := s.repo.UserByID(loadCtx, tenantID, userID)
if err != nil {
return User{}, err
}
s.cache.Set(tenantID, userID, user, time.Minute)
return user, nil
})
select {
case <-ctx.Done():
return User{}, ctx.Err()
case result := <-resultCh:
if result.Err != nil {
return User{}, result.Err
}
user, ok := result.Val.(User)
if !ok {
return User{}, ErrUnexpectedResult
}
return user, nil
}
}The initial cache lookup keeps ordinary hits away from singleflight.Group. There is little point taking its mutex for every successful read.
The second lookup is less obvious. Suppose requests A and B both miss the cache. A enters singleflight, loads the row and fills the cache. A finishes just before B calls DoChan. There is no longer an in-flight call for B to join, so B becomes a new leader. Without the second check it performs another database query even though the cache is now populated.
Actually, this bit is interesting because singleflight suppresses overlapping executions, not logically redundant executions. The second cache check closes the small gap between those two ideas.
Cancellation belongs to callers, not the cohort
The simpler Group.Do blocks until the shared function returns. That is often fine for background jobs, but an HTTP request may disappear while another caller still wants the result.
DoChan lets each caller select between its own context and the shared result. The package documentation notes that the returned channel is not closed, so receive one result rather than ranging over it.
There is another cancellation trap. If the shared function uses the first caller's context directly, that caller owns the database operation. Its deadline expiring causes every waiter to receive the resulting error, even if their requests remain healthy.
The example uses context.WithoutCancel to preserve context values while detaching the operation from the first request's cancellation and deadline. It then applies a separate two-second database timeout. A cancelled waiter can leave immediately, while the shared load continues for other callers. If every waiter leaves, the query can still run until that independent timeout, which is why the timeout should be deliberate rather than enormous.
Context values are not magic metadata transport, incidentally. Tenant identity and anything affecting the query belong in explicit arguments and in the singleflight key. Trace information may reasonably remain in the context.
The key is part of your security boundary
Singleflight returns the leader's value and error to every duplicate caller. A key containing only userID would therefore be wrong if user IDs are scoped by tenant. Two simultaneous requests for tenant A's user 42 and tenant B's user 42 could be merged, leaking the result across tenants.
The example includes both tenant and user identifiers in an unambiguous length-prefixed key. Include every input that can change the result: tenant, locale, permissions, projection, database region and sometimes feature flags. Do not include irrelevant request details, or almost nothing will coalesce.
The returned Shared field can help with metrics, but it does not mean "this caller was a follower". It reports whether the result was supplied to more than one caller, so the leader can also receive Shared: true.
Errors get a flight too
Successful values are not special. If the database call fails, all callers waiting on that key receive the same error. That is generally preferable to turning one overloaded database timeout into another burst of immediate retries.
Once the call finishes, however, singleflight forgets it. The next request can try again straight away. It is not an error cache, a negative cache or a circuit breaker. If repeated failures need a cooling-off period, store a carefully chosen negative result in the cache or use a circuit breaker separately. Be particularly cautious about caching transient failures and permission-dependent "not found" responses.
Group.Forget(key) also does less than its name might suggest. It allows a future call for that key to start new work instead of joining the existing call. It does not cancel the function already running. Calling it during a slow query can therefore restore the duplicate load that singleflight was meant to prevent.
Where the protection stops
A Group lives in one Go process. Ten replicas can still produce ten database queries for the same cold key. That may be an excellent reduction from thousands of queries, but it is not cluster-wide coordination.
For protection across replicas, the usual options are a shared cache with sensible expiry jitter, serving stale values while one worker refreshes, or a distributed locking scheme whose failure behaviour has been thought through. Singleflight remains useful inside each replica and is considerably cheaper than acquiring a network lock for every miss.
Finally, every waiter receives the same value. Returning a User containing maps, slices or pointers and then letting callers mutate it can create data races or surprising cross-request changes. Treat shared results as immutable, or clone them before returning.
Used within those boundaries, singleflight is pleasingly small: it does not replace the cache, impose an eviction policy or pretend one process is a distributed system. It merely turns simultaneous identical work back into one piece of work, which is exactly what the database wanted all along.