Write Files Without a Cleanup Race with O_TMPFILE

Writing through file.tmp, then renaming it into place, is usually fine. The awkward bit is everything that happens before the rename: the temporary pathname exists, another process can see it, and a crash can leave debris behind.

Linux has a neat answer. O_TMPFILE creates the inode before it creates the name, so a file can be written, checked and synchronised while it is completely unreachable through the filesystem namespace.

The race is in the pathname

A conventional temporary-file sequence looks like this:

1. Create a uniquely named file in the destination directory.
2. Write the contents and handle failures.
3. Rename the temporary name to its final name.

Using an exclusive create, such as Go's os.CreateTemp, avoids the old predictable-name symlink attack. That is good, but the pathname still exists during the write. A directory watcher can observe it, another process can open it, and cleanup logic has to account for every failure path.

Quick detour: the cleanup problem is not merely untidy. A service may be killed between creating the temporary file and registering it for cleanup. If it writes sensitive data, that abandoned file can also outlive the operation that created it.

O_TMPFILE reverses the order

With O_TMPFILE, the kernel creates an unnamed regular file inside a specified directory. You receive a file descriptor, but there is no directory entry to find. If the process closes the descriptor without publishing it, the inode disappears.

The publication step is an explicit linkat call. Until that call succeeds, readers cannot open the eventual filename because it does not exist. After it succeeds, the name points at the fully written inode.

The Linux open(2) documentation describes this as one of the main uses of the flag. The pathname passed to open is a directory, not the file's eventual name.

Do not combine O_TMPFILE with O_EXCL if you intend to publish the file. Here, O_EXCL has a special meaning: it prevents the unnamed inode from being linked into the filesystem at all.

A Go implementation for a new file

Go's standard library does not expose this Linux-specific operation directly, so the example uses the maintained golang.org/x/sys/unix package. It creates a file, writes all bytes, flushes the file, publishes it under the requested name, then flushes the directory.

package tmpfile

import (
	"errors"
	"fmt"
	"os"
	"path/filepath"

	"golang.org/x/sys/unix"
)

// WriteNew publishes path only after data has been written and synced.
// The destination must not already exist.
func WriteNew(path string, data []byte) error {
	dir, name := filepath.Split(path)
	if name == "" {
		return fmt.Errorf("path has no filename: %q", path)
	}
	if dir == "" {
		dir = "."
	}

	dirfd, err := unix.Open(dir, unix.O_RDONLY|unix.O_DIRECTORY|unix.O_CLOEXEC, 0)
	if err != nil {
		return fmt.Errorf("open destination directory: %w", err)
	}
	defer unix.Close(dirfd)

	fd, err := unix.Openat(
		dirfd,
		".",
		unix.O_WRONLY|unix.O_TMPFILE|unix.O_CLOEXEC,
		0600,
	)
	if err != nil {
		return fmt.Errorf("open unnamed temporary file: %w", err)
	}

	file := os.NewFile(uintptr(fd), "unnamed temporary file")
	if file == nil {
		unix.Close(fd)
		return errors.New("create Go file wrapper")
	}
	defer file.Close()

	if _, err := file.Write(data); err != nil {
		return fmt.Errorf("write temporary file: %w", err)
	}
	if err := file.Sync(); err != nil {
		return fmt.Errorf("sync temporary file: %w", err)
	}

	// AT_EMPTY_PATH refers to the inode through fd. It may require
	// CAP_DAC_READ_SEARCH, so use /proc/self/fd as the unprivileged fallback.
	err = unix.Linkat(fd, "", dirfd, name, unix.AT_EMPTY_PATH)
	if errors.Is(err, unix.EPERM) {
		procPath := fmt.Sprintf("/proc/self/fd/%d", fd)
		err = unix.Linkat(
			unix.AT_FDCWD,
			procPath,
			dirfd,
			name,
			unix.AT_SYMLINK_FOLLOW,
		)
	}
	if err != nil {
		return fmt.Errorf("publish %q: %w", name, err)
	}

	if err := unix.Fsync(dirfd); err != nil {
		return fmt.Errorf("sync destination directory: %w", err)
	}
	return nil
}

The direct AT_EMPTY_PATH form is the cleanest way to link the open file descriptor. On systems where the process lacks the capability required by linkat, the /proc/self/fd form asks the kernel to follow the descriptor's procfs link instead.

This is Linux-only code. The import should normally sit behind a Linux build constraint if the package also needs to compile on other operating systems.

Why the two sync calls are separate

file.Sync() deals with the file's contents and inode metadata. It does not necessarily make the directory entry durable. Linux's fsync(2) documentation explicitly calls out the need to synchronise the directory separately.

The sequence therefore has two different promises:

• Before linking, the file's contents have been flushed as far as file.Sync() can guarantee.
• After linking and syncing the directory, the name-to-inode relationship has also been submitted for durable storage.

Atomic visibility and crash durability are related, but they are not the same guarantee. Without the file sync, a published name might survive while its contents do not. Without the directory sync, the contents may be safe while the new name disappears after a power failure.

A failed directory sync is especially awkward: publication already happened, so returning an error does not mean the file is absent. Callers that need strong recovery behaviour should record which stage failed and reconcile the directory on restart.

What this does, and what it does not do

For a new file, successful publication is a one-time transition. Readers see no pathname and then see the complete file. They never see the half-written state that a direct write to the final name would expose.

That makes this useful for generated manifests, cache entries, downloaded objects and other files where a failed operation should leave nothing behind.

It does not replace every atomic-write protocol. In particular, linkat will fail if the destination name already exists. That is often exactly what you want for create-only publication, but replacement needs a different sequence.

For replacement, one option is to link the unnamed inode under a private staging name and then use rename to replace the old name. The staging name is visible for a short time, and the directory needs another durability step. The rename(2) documentation covers the atomic replacement property, but it does not turn the whole operation into a transaction.

Check the filesystem before depending on it

Support for O_TMPFILE depends on the filesystem and kernel, not merely on the Linux distribution. An unsupported filesystem commonly reports EOPNOTSUPP. Network filesystems, container mounts and unusual storage layers deserve particular suspicion.

In practice, test the exact destination directory during installation or startup. A small probe can attempt to open an unnamed file, close it, and treat EOPNOTSUPP as a capability result rather than a mysterious application failure.

• Use the final destination directory, not a convenient temporary directory on another mount.
• Keep the unnamed file open until after the final linkat.
• Do not use O_EXCL when publication is required.
• Decide whether file and directory fsync are worth their storage cost for this data.

There is something pleasing about the design: cleanup is no longer a second problem bolted onto writing. The file becomes real only when the write has earned a name.