Debug a systemd Boot with Temporary Masks and Wants

This guide shows how to change one systemd boot transaction from the kernel command line. You will be able to stop a troublesome unit from being pulled in, add a diagnostic service, or open systemd's early debug shell without editing unit files. The changes are runtime-only and disappear at shutdown.

Allow about 15 minutes if you already have console access. You need root access to change the bootloader configuration or kernel command line, and a recovery path if the machine does not boot normally. The examples match the installed systemd 255.4 package on this machine.

What this generator reads

systemd-debug-generator is a systemd generator, not a command that you normally run by hand. During boot it reads the kernel command line and creates transient generator output for the current boot. The executable is installed at /usr/lib/systemd/system-generators/systemd-debug-generator.

There are three kinds of option. systemd.mask=UNIT masks a unit in the main system. systemd.wants=UNIT adds a start job to the initial transaction. systemd.debug_shell pulls in debug-shell.service. Each option can be repeated where a value is expected.

Options beginning with rd. apply to the initial RAM disk, or initrd: rd.systemd.mask=, rd.systemd.wants=, and rd.systemd.debug_shell. Their unprefixed counterparts apply only to the main system. This boundary is an easy place to lose time: a main-system mask does not mask an initrd unit, and an initrd option does not control a service started after the real root filesystem is handed over.

Checkpoint: identify the boot you are changing

1. Inspect the active kernel command line as an ordinary user:

   tr ' ' '\n' < /proc/cmdline

   Look for existing systemd. or rd.systemd. options before adding another copy. A bootloader entry may be generated from a distribution configuration, so record the exact entry you will edit.

Mask a unit for one boot

Use a mask when a unit is causing the boot failure or pulling in a dependency you need to exclude while investigating. Add this kernel argument to the selected boot entry:

systemd.mask=example.service

Replace example.service with the real unit name. You can add the argument temporarily in a bootloader's edit screen, or add it to a persistent bootloader configuration as a root change. If the failing unit is started in the initrd, use the corresponding initrd form instead:

rd.systemd.mask=example-initrd.service

After boot, verify the result with:

systemctl is-enabled example.service
systemctl status example.service --no-pager

A runtime mask normally reports masked from is-enabled, and status may say that the unit is masked. The mask affects this boot from startup to shutdown; it is not the same as permanently running systemctl mask.

Start a diagnostic unit at boot

Use a wants option when you need an existing service or one-shot unit to run as part of the initial boot transaction:

systemd.wants=example-diagnostic.service

For an initrd diagnostic, use:

rd.systemd.wants=example-initrd-diagnostic.service

These options request a start job. They do not create a unit, enable it permanently, or guarantee that it will succeed. Once the system is up, inspect the request and its logs:

systemctl status example-diagnostic.service --no-pager
journalctl -b -u example-diagnostic.service --no-pager

If the unit does not exist in the relevant environment, the request cannot provide useful diagnostics. Check the unit name and whether it is present in the initrd before relying on this method.

Open the early debug shell

To request a shell during early boot on the main system, add:

systemd.debug_shell

The service uses /dev/tty9 by default. You can specify a different terminal by assigning a value, with or without the /dev/ prefix:

systemd.debug_shell=tty3
systemd.debug_shell=/dev/tty3

Only use one of those forms in a boot entry. Switch to the selected virtual terminal after boot. The debug shell is a root shell and is intentionally powerful. Anyone who can reach that terminal can control the machine, so do not leave this option in a shared, unattended or production boot configuration.

The initrd variant is separate:

rd.systemd.debug_shell=tty3

That shell belongs to the initrd phase. It is useful for early storage, root discovery and initrd service failures, but it is not a replacement for a main-system debug shell.

Checkpoint: confirm and undo the change

After rebooting, confirm which arguments actually reached the kernel:

cat /proc/cmdline

Then check the transaction's evidence. For a masked service, use systemctl status. For a requested service, use its status and boot journal. If you changed a persistent bootloader configuration, remove the systemd.* or rd.systemd.* argument and regenerate that configuration using your distribution's documented bootloader procedure. If you supplied the argument only in a one-time boot edit, simply rebooting without it is the undo.

Runtime masks and wants do not survive shutdown. A persistent systemctl mask performed separately does survive, so do not confuse it with the generator's temporary mask. If the machine will not boot, use the bootloader's edit screen to remove the argument, or choose an older known-good entry. A debug shell may also expose a recovery route, but treat it as privileged emergency access.