Safely Enter an overlayroot Lower Filesystem with overlayroot-chroot

You will finish with a controlled way to enter the read-only lower filesystem beneath an active overlayroot mount, make a targeted change, and leave the temporary mounts cleaned up. The examples use overlayroot-chroot from Ubuntu's overlayroot package, version 0.49~24.04.1 on this machine.

Allow about fifteen minutes for a simple inspection, or longer if you need to repair a package or configuration. You need an active overlayroot filesystem, a root shell, and a clear recovery plan for any change made inside the lower filesystem. This is an administrative and security-sensitive operation: writes bypass the overlay's normal temporary or backing-device view and can affect the next boot.

1. Confirm that overlayroot is actually active

Do not start by guessing a lower-directory path. The wrapper discovers it from the live mount table. First inspect the root filesystem and the package version:

$ findmnt -no TARGET,FSTYPE,OPTIONS /
$ dpkg-query -W -f='${Package} ${Version}\n' overlayroot
overlayroot 0.49~24.04.1

For a working overlayroot setup, the first command should show an overlay filesystem mounted at /. The options normally include a lowerdir= value. The second command is a version check, not a configuration change.

Checkpoint: if / is an ordinary filesystem, stop here. On this host it is ext4, and /etc/overlayroot.conf leaves overlayroot empty, so the wrapper correctly refuses to continue:

$ sudo overlayroot-chroot /usr/bin/true
ERROR: Unable to find an overlayroot filesystem
$ printf 'exit status: %s\n' "$?"
exit status: 1

The command does not enable overlayroot. Enabling it is a boot and initramfs change, outside this guide. If the host should use overlayroot, review its deployment configuration and reboot through the normal maintenance process before trying again.

2. Choose a harmless command for the first entry

The command has one form: overlayroot-chroot [COMMAND [ARG]...]. Everything after the wrapper is passed to chroot. Begin with a read-only identity check rather than an interactive shell:

$ sudo overlayroot-chroot /usr/bin/sh -c 'printf "inside: "; findmnt -no TARGET,FSTYPE /; id -u'
INFO: Chrooting into [/path/to/lower]
inside: / overlay 0

The lower-directory path and the exact findmnt output vary by host. The useful checks are the informational line, a successful command, and user ID 0. The root identity is required by the chroot and mount operations, so use sudo or an already authorised root shell. Do not paste untrusted text into the shell command.

For an interactive session, use an explicit shell and remember that it starts in the lower filesystem, not in the overlay view you normally see:

$ sudo overlayroot-chroot /bin/sh
# printf '%s\n' 'lower filesystem'
# exit

A shell prompt is not proof that the lower filesystem is writable. The wrapper has attempted the remount before invoking chroot; verify the specific path you intend to change.

3. Make one reviewed change

Before writing, identify the exact file and make a backup on the lower filesystem. This example uses a placeholder path, so replace it only after checking that it is the intended file:

# TARGET=/etc/example.conf
# test -f "$TARGET" && cp --preserve=all "$TARGET" "$TARGET.overlayroot-chroot.bak"
# ls -l "$TARGET" "$TARGET.overlayroot-chroot.bak"
# vi "$TARGET"

That backup changes state and should be treated as part of the maintenance record. If the edit is wrong, restore it before leaving:

# cp --preserve=all /etc/example.conf.overlayroot-chroot.bak /etc/example.conf
# rm -- /etc/example.conf.overlayroot-chroot.bak

Do not run broad cleanup commands, package upgrades, or filesystem formatting commands in this shell merely because they are available. A change in the lower filesystem can alter the system after the overlay is removed, and a service may still be running against the overlaid view. Schedule service disruption separately and keep the change narrow.

4. Exit normally and verify cleanup

Use exit or let the child command finish. The wrapper binds the host's /proc, /run, and /sys into the lower directory when needed. It then unmounts those bind mounts and remounts the lower filesystem read-only before returning.

# exit
INFO: Chrooting into [/path/to/lower]
$ printf 'wrapper status: %s\n' "$?"
wrapper status: 0
$ findmnt -no TARGET,FSTYPE,OPTIONS /

The informational line is normally printed before the child command starts, so it can appear before the shell prompt or command output. The lower path is host-specific. Check that the original root view is back and that no temporary bind mount remains under it:

$ findmnt -rn -t overlay
$ mountpoint /proc /run /sys

The first command should show the host's expected overlay mounts, if any. The second command checks the host paths, not the hidden lower-directory bind mounts. If the wrapper reports an unmount or remount error, do not reboot or start more maintenance blindly. Record the error, inspect the mount table as root, and restore any changed file from its backup.

5. Diagnose the common stopping points

• Unable to find an overlayroot filesystem: the root mount table does not contain an overlayroot entry. Check findmnt /, the kernel command line and the overlayroot configuration. Installing the package alone does not create an active overlay.
• Unable to find the overlayroot lowerdir: the mount entry names a lower directory that is not itself mounted. Stop and investigate the mount layout instead of supplying a guessed directory.
• Unable to bind /proc, /run or /sys: the lower filesystem does not have the expected mount point or the mount operation was denied. This is a prerequisite failure, not permission to edit a different path.
• Unable to remount writable: the filesystem, kernel or mount arrangement rejected the transition. The wrapper exits and attempts cleanup. Check its stderr and findmnt; do not assume a partial change is safe.
• The child command fails: chroot still needs the requested executable and its runtime dependencies inside the lower filesystem. Try an absolute path such as /bin/sh, and do not confuse a missing lower-tree binary with a mount failure.

Signals and failed child commands trigger the wrapper's cleanup trap. Cleanup itself can fail, however, so always perform the post-exit mount check. If a lower filesystem remains writable, remounting it read-only is a privileged recovery action and should be done only after confirming the exact target with findmnt. Never remount a device by guessed name.