Blog / Go

963 words

  • go
  • net-http
  • slog
  • tls
  • logging
  • debugging

Go's http.Server ErrorLog: Catching TLS Handshake Noise in slog

Put a Go server on a public port 443 and, within about an hour, your structured JSON logs gain a few lines that look like they escaped from 2009:

2026/09/26 09:14:02 http: TLS handshake error from 203.0.113.7:51544: EOF
2026/09/26 09:14:07 http: TLS handshake error from 198.51.100.22:40312: tls: first record does not look like a TLS handshake

No level, no fields, a timestamp format your log pipeline never asked for, and they go to stderr regardless of what you did with slog.SetDefault. This is http.Server.ErrorLog at work, or rather the fallback when you have not set it.

What ErrorLog actually catches

ErrorLog is a *log.Logger. If it is nil, net/http uses the standard library's global log package. It is not just for TLS. The server uses it for:

  • TLS handshake failures ("http: TLS handshake error from ...")
  • Accept errors it retries after (for example running out of file descriptors)
  • Handler panics, with the stack trace, in a single multi-line message ("http: panic serving ...")
  • "superfluous response.WriteHeader call" complaints
  • HTTP/2 connection-level oddities, which start with "http2:" rather than "http:"

Everything arrives as one pre-formatted string. There is no severity, so a panic with a stack trace and a port scanner hanging up mid-handshake are the same to a naive adapter. That is the real problem: not the format, but that a genuine incident and background radiation share one channel.

The one-liner, and why it is not enough

Since Go 1.21, slog.NewLogLogger gives you a *log.Logger that feeds a slog.Handler at a fixed level:

srv := &http.Server{
	Addr:     ":8443",
	Handler:  mux,
	ErrorLog: slog.NewLogLogger(logger.Handler(), slog.LevelError),
}

Now the lines are JSON, they go where your other logs go, and they carry your handler's attributes. But every record is at Error, and the message is still the whole opaque sentence. An alert on "error rate" now fires every time someone runs a scanner against you. Fixing that means looking inside the string.

A small writer that classifies

A log.Logger just needs an io.Writer, so give it one that understands the handshake line. The format is fixed in net/http as "http: TLS handshake error from <remote>: <reason>".

package main

import (
	"context"
	"log"
	"log/slog"
	"net/http"
	"os"
	"strings"
)

// errorLog adapts the *log.Logger that net/http wants onto slog.
type errorLog struct{ l *slog.Logger }

func (e errorLog) Write(p []byte) (int, error) {
	msg := strings.TrimSuffix(string(p), "\n")
	ctx := context.Background()

	rest, ok := strings.CutPrefix(msg, "http: TLS handshake error from ")
	if !ok {
		e.l.LogAttrs(ctx, slog.LevelError, "http server error",
			slog.String("detail", msg))
		return len(p), nil
	}

	remote, reason, _ := strings.Cut(rest, ": ")
	level := slog.LevelWarn
	switch {
	case reason == "EOF",
		strings.HasPrefix(reason, "tls: first record does not look like"),
		strings.HasPrefix(reason, "client sent an HTTP request"):
		level = slog.LevelDebug
	}
	e.l.LogAttrs(ctx, level, "tls handshake failed",
		slog.String("remote", remote),
		slog.String("reason", reason))
	return len(p), nil
}

func main() {
	logger := slog.New(slog.NewJSONHandler(os.Stderr, nil))
	mux := http.NewServeMux()

	srv := &http.Server{
		Addr:     ":8443",
		Handler:  mux,
		ErrorLog: log.New(errorLog{logger}, "", 0),
	}
	logger.Error("server stopped",
		"err", srv.ListenAndServeTLS("cert.pem", "key.pem"))
}

Three details that will bite if you skip them:

  • The log.New flags must be 0 and the prefix empty. Leave the default date flags on and your CutPrefix never matches, because the timestamp is now at the front of the line.
  • Return len(p) and a nil error. A short write makes log.Logger swallow the error silently, but returning the wrong count is a bug you will not enjoy finding later.
  • Anything that is not a handshake line goes out at Error with the whole text in one attribute. That keeps a panic's stack trace intact instead of scattering it across records.

What is noise and what is not

This is where I would not just demote everything. Roughly, the handshake failures fall into three groups.

Background radiation. "EOF" is a client that opened a TCP connection and hung up before sending a ClientHello: port scanners, load balancer health checks doing a bare TCP probe, the odd impatient mobile client. "tls: first record does not look like a TLS handshake" is somebody speaking something other than TLS to your port. Both are safe at Debug. Plain HTTP sent to the HTTPS port is a bit more interesting (actually, a bit of a favourite of mine): the server answers it with a 400 saying "Client sent an HTTP request to an HTTPS server" and then logs it with that reason. Usually a misconfigured client or an old redirect, so Debug is fine, though a sudden spike is worth a glance.

Clients rejecting you. Reasons starting "remote error: tls:" are alerts the client sent to us. "unknown certificate authority" or "bad certificate" means the client looked at your certificate and refused it. If it is one curl user with a self-signed cert, fine. If it is every client at once, your certificate has expired or your chain is incomplete, and this is the only server-side trace you will get. I keep these at Warn on purpose. Demoting the whole "remote error" family to save log volume would hide the one failure that looks exactly like your own outage.

Negotiation failures. "client offered only unsupported versions" and "no cipher suite supported by both client and server" are usually old clients or scanners probing for legacy protocols. Warn is a reasonable default; move them to Debug once you have looked at who is sending them.

You can reproduce the main cases without waiting for the internet to oblige. curl http://localhost:8443/ hits the plain-HTTP path, and curl https://localhost:8443/ with a self-signed certificate should produce a client alert about the CA. Run the server with the handler at Debug level while you try it.

The string is not an API

Everything above depends on message text that the Go authors are free to reword. The prefix has been stable for a long time, but the wording of the reasons is partly the crypto/tls error strings, and those do change between releases. Two cheap defences. First, make the default the loud one (as here: unknown means Warn or Error, and only known-boring strings are demoted), so a wording change makes you noisier rather than blinder. Second, add a test that starts an httptest TLS server with your ErrorLog, writes a few bytes of garbage to the raw socket, and asserts the classification. It fails on a Go upgrade, which is when you want to hear about it.

One last thing about the mechanics. log.Logger takes a mutex around each write, so your writer is called serially. If the sink behind your slog handler is slow (a blocking network log shipper, say), a burst of failed handshakes queues up behind it, each one holding a connection goroutine. During a scan that is exactly when you have plenty of them. Keep the writer cheap, and buffer or drop at the handler layer rather than blocking here.

← Back to the blog