Debug and Safely Replay udev Events with udevadm

udevadm lets you interrogate the udev database, test a rule before it goes live, and replay a device event without waking the whole system up. These examples target the installed systemd 255 build. Allow about 10 minutes. You need a shell; several commands that change daemon state or replay events require root.

Before you begin

Keep a second terminal open if you are chasing a device being added or removed. Do not start with a system-wide trigger: it asks the kernel to re-emit events, which can run rules and cause real device changes. Start with --dry-run, then narrow to a subsystem or device match.

The examples use /dev/null because every Linux box has one. Swap in the device you actually need to investigate, such as /dev/sdb, and check the path carefully before running anything privileged.

1. Inspect the device database

Query the udev database with a device node, a /sys/ path, or a systemd device unit. This asks udev what it already knows; it does not replay an event.

udevadm info --query=property --property=DEVNAME,DEVPATH /dev/null
udevadm info --query=property --property=DEVNAME --value /dev/null

The first command prints named properties: on this machine, DEVPATH=/devices/virtual/mem/null and DEVNAME=/dev/null. The second prints only /dev/null. The --value form is handy in scripts, but only with a property query, not with --export.

For rule writing, udevadm info --attribute-walk /dev/null prints matching sysfs attributes up the device chain. Treat that output as clues for a rule, not a shopping list of every attribute to copy in. Prefer stable identifiers and test the rule you end up with.

2. Validate a rule before installing it

udevadm verify checks udev rules for syntactic, semantic and stylistic errors, and takes one or more files. This harmless example checks a temporary file and never installs or loads it:

rule_file=$(mktemp /tmp/udevadm-guide-XXXX.rules)
printf '%s\n' 'SUBSYSTEM=="misc", KERNEL=="null", TAG+="udevadm-guide-example"' > "$rule_file"
udevadm verify "$rule_file"
rm -- "$rule_file"

Verification is not activation. A valid rule still has to land in a udev rules directory, and the daemon has to reload it. Reloading does not retroactively apply new rules to devices that already exist.

3. Simulate one event

udevadm test simulates rule processing for one sysfs device. It prints debug output and does not run RUN programs, which makes it good for watching properties and rule decisions, but its output can still differ from a real event.

udevadm test --action=add /sys/devices/virtual/mem/null

Look near the end for properties such as DEVPATH, DEVNAME, MAJOR and MINOR. The command itself reports that it is for debugging only, and a failed simulation is not proof the real device is broken: permissions, timing and event context can all differ.

To test a built-in operation, list the supported built-ins first, then pass a command and a device path:

udevadm test-builtin --help
udevadm test-builtin hwdb /sys/devices/virtual/mem/null

Use the exact device path and built-in name your installed command shows you. That list is version-specific.

4. Reload rules, then replay narrowly

After installing a corrected rule, reload the daemon's rules and related databases. This needs elevated privileges:

sudo udevadm control --reload

Check the daemon is actually there, separately:

sudo udevadm control --ping

A successful ping gives no useful output but exits cleanly. A permission error just means the command could not reach the daemon from your current context; it does not prove the daemon is stopped.

Preview a scoped replay before you fire it. This lists the matching device without sending the event:

udevadm trigger --dry-run --verbose --subsystem-match=misc --name-match=/dev/null

Only once that list is exactly what you intend, replay the change event and wait for it to finish:

sudo udevadm trigger --subsystem-match=misc --name-match=/dev/null --action=change --settle

5. Wait for the result

udevadm settle waits for the whole current event queue, with a default timeout of 120 seconds. trigger --settle only waits for events that trigger itself created. If a script needs one device to exist and be initialised, reach for udevadm wait instead:

udevadm wait --timeout=10 /dev/null
udevadm settle --timeout=0

The first line checks one device and fails if it is not ready within 10 seconds. The second is an immediate queue check: a zero timeout never waits.

Where iocost fits

On this installation, /usr/lib/udev/rules.d/90-iocost.rules applies an I/O cost solution to block disks when the udev property IOCOST_SOLUTIONS is present. /etc/systemd/iocost.conf can set TargetSolution=, added in systemd 254. Prefer an administrator drop-in such as /etc/systemd/iocost.conf.d/50-local.conf over editing the vendor file: drop-ins are read in lexical order, with later values overriding earlier single-value settings.

Inspect the device's udev properties and available metadata before picking a target solution. A solution missing from a device falls back to the first one listed in IOCOST_SOLUTIONS. This configuration affects I/O isolation, so test it during a maintenance window.

Done means

• Queried the target device. udevadm info told you what udev already knows about it.
• Passed verification. Your rule cleared udevadm verify before installation.
• Simulated safely. udevadm test showed you rule processing without running any RUN commands.
• Previewed before triggering. You checked the matched device list and scoped it by subsystem or device before touching root.
• Confirmed the result. You waited for the relevant device or event queue and recorded what happened.