Articles / Linux

  • linux
  • file-descriptors
  • fork-exec
  • cloexec
  • shell
  • debugging

Why a Unix Process Inherits File Descriptors You Forgot About

Run this and watch it hang for a minute, even though the shell command finished instantly:

sh -c 'sleep 60 &' | cat

The shell exits at once, but cat sits there waiting. The reason is that sleep inherited the write end of the pipe as its standard output, so the pipe has a writer until sleep dies. A child holds every descriptor you did not explicitly close or mark for closing.

What fork and exec actually keep

The rules are short, and they are the whole story:

  • fork() copies the parent's descriptor table. Child and parent now point at the same open file descriptions.
  • execve() replaces the program but keeps the descriptor table, except for descriptors with the close-on-exec flag (FD_CLOEXEC) set.
  • Descriptor 0, 1 and 2 are just the first three entries. Nothing special happens to descriptor 7.

So if your program opens a config file, a database socket or a secret, and then runs system() or spawns a helper, that helper can read the lot unless something closed it.

Seeing it for yourself

The shell makes this easy to demonstrate. Open a descriptor, then list what a child sees:

exec 9</etc/hostname
ls -l /proc/self/fd

The ls output includes 9 -> /etc/hostname, alongside the descriptor ls itself opened to read the directory. The shell did not tell ls about it. It simply never closed it.

For a process that is already running, ls -l /proc/<pid>/fd or lsof -p <pid> shows the same thing. Compare a parent and its children and the odd ones out are your leaks.

Why the pipe hangs

EOF on a pipe only arrives when every write end is closed. That includes copies held by processes you never think about.

Quick detour, because this one bites people in CI. A build step starts a background daemon from a script, the script exits, and the CI runner waits forever for the output stream to close. The daemon inherited stdout, so as far as the runner can tell, the job is still writing. The fix is to redirect the daemon's descriptors at launch:

sleep 60 >/dev/null 2>&1 </dev/null &

Now the pipe has no stray writers and cat returns as soon as the shell does.

The nastier cases

Hanging output is the polite failure. Worse ones:

  • A listening socket leaks into a child. You restart the parent, but a long-lived child still holds the socket, so the port stays bound and the new process gets "address already in use".
  • A lock outlives its owner. A flock belongs to the open file description, so a child holding a copy keeps it held.
  • A secret file is readable by code you did not mean to trust. A plugin or helper binary inherits an open descriptor to a key file and can read() it without ever knowing its path.
  • A deleted file keeps eating disk space because a forgotten child still has it open.

The third one is a security problem rather than a tidiness problem. Dropping privileges or changing directory does nothing about descriptors you already hold.

The fix: close-on-exec, set at open time

Pass O_CLOEXEC to open(), SOCK_CLOEXEC to socket(), and pipe2(..., O_CLOEXEC) for pipes. Setting the flag afterwards with fcntl(fd, F_SETFD, FD_CLOEXEC) works, but there is a gap between the two calls. In a threaded program another thread can fork and exec inside that gap, and the descriptor leaks anyway. Setting it atomically at creation closes the race.

Most modern runtimes already do this for you:

  • Go: the standard library opens files and sockets close-on-exec. os/exec hands the child only descriptors 0 to 2 plus whatever you list in Cmd.ExtraFiles, which become 3 onwards.
  • Python 3.4 and later: descriptors are non-inheritable by default (PEP 446), and subprocess uses close_fds=True.
  • Rust: std opens files with close-on-exec set, and Command inherits only the standard streams unless you do something unusual.

The leaks therefore tend to come from C code, shell scripts, descriptors handed over by a supervisor, and libraries that call open() directly.

Closing everything else before exec

If you are writing the launcher yourself and cannot trust what is open, close the range between fork and exec. On Linux 5.9 and later, close_range(2) does it in one call:

/* in the child, after fork(), before execve() */
if (close_range(3, ~0U, 0) == -1) {
    _exit(127);
}

Linux 5.11 added the CLOSE_RANGE_CLOEXEC flag, which marks the range close-on-exec instead of closing it immediately. That is handy when you still need a descriptor to report an exec failure back to the parent. Older kernels need a loop over /proc/self/fd, which is what many libraries still do.

A short audit routine

  1. Start the process, then run ls -l /proc/<pid>/fd on it.
  2. Run a child from it and list that child's descriptors the same way.
  3. Anything beyond 0, 1 and 2 that you did not intend to pass is a leak. Find where it was opened and add the close-on-exec flag there.

On the shell side, exec 9>&- closes a descriptor you opened, and redirecting a background job's streams stops it holding your pipes open. When something refuses to see EOF or refuses to release a port, check who else holds a copy before blaming the code that looks guilty.