SCM_RIGHTS and File Descriptor Passing: The Unix Socket Trick systemd Uses to Hand Off Privileged Sockets

Here is a question that sounds like a trick until you have seen the answer: can one process open a file descriptor and then physically move it into a completely unrelated process, one it never forked and doesn't share memory with? Not a copy of the data, not a path the other process can reopen itself, the actual open file descriptor, pointing at the same open file description in the kernel. Yes, and the mechanism is older than most of the software that relies on it: SCM_RIGHTS, an ancillary data type you can send down a Unix domain socket.

This matters more than it looks like it should, because it is the standard answer to a recurring privilege-separation problem: something needs to bind port 443, or open /dev/kvm, or hold a raw socket, but you don't want the process that actually handles untrusted input running with the capability to do that. The classic fix is to split it into two processes: a small, trusted one that holds the privilege and does the one privileged thing, and a larger, unprivileged one that does everything else. The privileged fd has to cross that boundary somehow, and it can't cross it as a number, because file descriptor numbers are only meaningful within a single process's fd table. SCM_RIGHTS is how it crosses.

What actually gets sent

A Unix domain socket can carry two kinds of payload on every sendmsg/recvmsg call: the ordinary byte stream (or datagram), and a side channel of "ancillary data" (also called control messages, or cmsgs). SCM_RIGHTS is a control message type at the SOL_SOCKET level whose payload is an array of file descriptor numbers, as understood by the sender. The kernel does not send these numbers verbatim: it walks the array, and for each one, installs a new descriptor in the receiving process's fd table that refers to the same open file description as the sender's original. When the receiver reads the message, it gets back a (probably different) integer, but that integer now behaves exactly as if the receiver had opened the file, socket, or pipe itself: same read/write offset if it's a regular file, same socket state if it's a connected TCP socket, same everything.

This is a genuinely different operation from dup2, because dup2 only works within one process. SCM_RIGHTS is dup2 across a process boundary that the two ends don't otherwise share. The only requirement is that both ends have an open Unix domain socket connected to each other; they don't need to be related by fork, they don't need to share a filesystem namespace for the fd's target, they just need that one socket.

A minimal working example

Go's standard library has everything needed for this without a single third-party dependency: syscall.Socketpair to get a connected pair of Unix domain sockets, and syscall.UnixRights/ParseUnixRights to build and parse the control message. net.UnixConn exposes WriteMsgUnix and ReadMsgUnix, which take the ancillary data as an explicit parameter, so nothing here is hidden behind a higher-level abstraction that would obscure what's happening.

package main

import (
	"fmt"
	"net"
	"os"
	"syscall"
)

func fdToUnixConn(fd int) (*net.UnixConn, error) {
	f := os.NewFile(uintptr(fd), "socketpair")
	defer f.Close()
	conn, err := net.FileConn(f)
	if err != nil {
		return nil, err
	}
	return conn.(*net.UnixConn), nil
}

func main() {
	fds, err := syscall.Socketpair(syscall.AF_UNIX, syscall.SOCK_STREAM, 0)
	if err != nil {
		panic(err)
	}

	sender, err := fdToUnixConn(fds[0])
	if err != nil {
		panic(err)
	}
	defer sender.Close()

	receiver, err := fdToUnixConn(fds[1])
	if err != nil {
		panic(err)
	}
	defer receiver.Close()

	// Anything with a file descriptor works here: an open file, a
	// listening TCP socket bound to port 443, a raw socket, a pipe end.
	f, err := os.Open("/etc/hostname")
	if err != nil {
		panic(err)
	}
	defer f.Close()

	rights := syscall.UnixRights(int(f.Fd()))
	if _, _, err := sender.WriteMsgUnix(nil, rights, nil); err != nil {
		panic(err)
	}

	oob := make([]byte, syscall.CmsgSpace(4))
	_, oobn, _, _, err := receiver.ReadMsgUnix(nil, oob)
	if err != nil {
		panic(err)
	}

	cmsgs, err := syscall.ParseSocketControlMessage(oob[:oobn])
	if err != nil {
		panic(err)
	}
	recvFds, err := syscall.ParseUnixRights(&cmsgs[0])
	if err != nil {
		panic(err)
	}

	received := os.NewFile(uintptr(recvFds[0]), "received")
	defer received.Close()

	data := make([]byte, 128)
	n, _ := received.Read(data)
	fmt.Printf("read via handed-off fd: %s", data[:n])
}

Nothing here required the receiver to have opened /etc/hostname, or even to have permission to open it independently. It only had permission to receive an already-open descriptor to it. That distinction is the entire point of the pattern, and also its main security consideration: possession of the Unix socket is effectively possession of whatever capability gets sent across it, so the socket's own permissions (filesystem mode bits, or peer credential checks via SO_PEERCRED) become the actual access control boundary. Get those wrong and you've built a very elaborate way to hand root-owned sockets to anyone who can connect.

One practical limit worth knowing: Linux caps the number of descriptors you can pass in a single message at SCM_MAX_FD, 253 on current kernels. It rarely bites in practice, since most uses pass one or two fds at a time, but it explains why nobody tries to bulk-transfer an entire fd table this way.

Where systemd actually uses this, and where it doesn't

systemd's socket activation is the thing most people associate with fd handoff, and it's worth being precise about what it actually is, because it isn't SCM_RIGHTS. When a .socket unit is triggered, systemd opens the listening socket itself, then forks and execs the corresponding service with that socket's fd still open (starting at fd 3, with $LISTEN_FDS and $LISTEN_PID set so the service, typically via sd_listen_fds(), knows it's there). That's plain fd inheritance across fork/exec, the same mechanism that keeps stdin/stdout/stderr open across any process spawn. No ancillary data involved, because systemd is the direct parent of the process it's handing the fd to.

The genuinely interesting use of SCM_RIGHTS in systemd shows up somewhere less obvious: the file descriptor store. A service that wants its listening sockets to survive a crash or restart without a gap in which connections get refused can call sd_pid_notify_with_fds(), sending an FDSTORE=1 message plus the fds themselves over $NOTIFY_SOCKET, an AF_UNIX SOCK_DGRAM socket connecting the service back to the systemd manager. Because systemd and the service are not parent and child at that point in any way that makes inheritance available, the only way to get the fd across is exactly the mechanism above: a control message on a Unix socket. systemd holds onto the descriptor, and if the service restarts, hands it straight back on the next start, so the listening socket never actually closes, only the process holding it changes.

The same trick shows up under different names all over privilege-separated software: OpenSSH's privilege separation model, D-Bus handing fds to bus clients so they don't need to open device nodes themselves, and Chromium's sandboxed renderer processes receiving already-opened files from a broker process that does the actual filesystem access on their behalf. In every case the shape is identical: a trusted process decides what gets opened, an untrusted or lower-privileged one gets to use it without ever having had the ability to open it itself. The socket isn't carrying data at that point, it's carrying a capability.