Articles / Go

770 words

  • go
  • tcp
  • net-listener
  • denial-of-service
  • networking
  • security

Why Your Go TCP Server Needs a Connection Cap First

A goroutine per connection is fine right up until the connections are the problem. The goroutine costs a couple of kilobytes; the connection costs a file descriptor, kernel socket buffers, and whatever your handler allocates for it. Nothing in a plain accept loop says "no", so the first limit you hit is whichever one the operating system picks for you.

What an unbounded accept loop actually does

Here is the shape most Go TCP servers start with:

for {
	c, err := ln.Accept()
	if err != nil {
		log.Print(err)
		continue
	}
	go handle(c)
}

Every client that completes a handshake gets a goroutine, a descriptor and a buffer. Ten thousand idle clients cost you ten thousand of each, and a hostile client can open them without sending a byte.

Worse, look at that continue. When descriptors run out, Accept fails with EMFILE immediately, every time, and the loop spins at full CPU logging the same error. The server is now both out of capacity and busy doing nothing useful.

Where the real limits sit

  • File descriptors: the per-process limit (ulimit -n), which the Go runtime raises to the hard limit at startup in recent releases, but which is still finite.
  • Memory: per-connection buffers, usually the dominant cost, and multiplied by your cap rather than by your goroutine count.
  • Downstream resources: a database pool of 20 does not care that you accepted 5,000 clients.

A cap lets you do the arithmetic once: connections times worst-case memory per connection, compared with what the box has. Without a cap, that sum is whatever an attacker makes it.

A cap in thirty lines of stdlib

The trick is to take a slot before calling Accept, not after. Then at capacity the loop simply blocks, and the kernel's listen backlog absorbs the queue instead of your heap.

type limitListener struct {
	net.Listener
	slots chan struct{}
}

func newLimitListener(l net.Listener, n int) net.Listener {
	return &limitListener{Listener: l, slots: make(chan struct{}, n)}
}

func (l *limitListener) Accept() (net.Conn, error) {
	l.slots <- struct{}{} // blocks while we are at the cap
	c, err := l.Listener.Accept()
	if err != nil {
		<-l.slots
		return nil, err
	}
	return &limitConn{
		Conn:    c,
		release: sync.OnceFunc(func() { <-l.slots }),
	}, nil
}

type limitConn struct {
	net.Conn
	release func()
}

func (c *limitConn) Close() error {
	err := c.Conn.Close()
	c.release()
	return err
}

The sync.OnceFunc matters. Handlers close connections in several places (a defer, an error path, a timeout), and releasing a slot twice would quietly raise your cap above what you configured.

If you would rather not write it, netutil.LimitListener in golang.org/x/net does the same thing. It is one dependency for about thirty lines, so it is your call.

Quick detour: where do the waiting clients go?

Hang on, if we stop calling Accept, what happens to clients that arrive? The kernel completes the TCP handshake for them and parks them in the listener's accept queue, up to the backlog size. Past that, new SYNs are dropped or ignored and clients retry.

You can watch it happen. Run ss -ltn and look at your listening port: for a listening socket, Recv-Q is the current accept queue length and Send-Q is its maximum. A Recv-Q that climbs while your process sits at the cap is the cap working.

The backlog is limited by the lower of the value Go passes and net.core.somaxconn on Linux. Go reads that sysctl at startup, so you do not normally need to touch it.

A cap with no deadlines is a lock-out

Here is the catch: a cap turns idle connections into a denial of service. If 1,000 clients connect and say nothing, you are full and real users queue behind them. The cap protects the process and hands the attacker a cheaper way to starve everyone else.

So every handler needs deadlines that free the slot. An idle timeout that resets after each successful read keeps long-lived connections alive without letting silent ones squat:

func handle(c net.Conn) {
	defer c.Close()
	buf := make([]byte, 4096)
	for {
		if err := c.SetReadDeadline(time.Now().Add(30 * time.Second)); err != nil {
			return
		}
		n, err := c.Read(buf)
		if n > 0 {
			if _, werr := c.Write(buf[:n]); werr != nil {
				return
			}
		}
		if err != nil {
			return
		}
	}
}

Setting the deadline once, before the loop, would kill healthy long-lived connections, because it is an absolute time rather than a timeout. Reset it each time round. You may want a write deadline too, for peers that stop reading.

Handle Accept errors properly

With the cap in place, tidy up the accept loop so it does not spin on a transient failure:

ln, err := net.Listen("tcp", ":7000")
if err != nil {
	log.Fatal(err)
}
ln = newLimitListener(ln, 1000)

for {
	c, err := ln.Accept()
	if errors.Is(err, net.ErrClosed) {
		return
	}
	if err != nil {
		log.Printf("accept: %v", err)
		time.Sleep(50 * time.Millisecond)
		continue
	}
	go handle(c)
}

The sleep is a crude backoff, and net/http does a similar thing internally for temporary accept errors. It keeps a descriptor shortage from turning into a log flood.

Choosing the number

Pick the cap from memory and downstream limits, not from how many goroutines Go can theoretically schedule. Measure resident memory with a few hundred real connections, divide, and leave headroom. Keep it comfortably below ulimit -n, since listeners, log files and outbound connections need descriptors too.

To test it, open a few more connections than the cap from a script and watch ss. The extras should sit in the queue, and your process's memory should stay flat. If you want a per-IP limit as well, that is a separate layer on top; the global cap is what keeps the process alive while you build it.

← All articles