Hand mdmon Back to the Real Root at Boot

mdmon watches external-metadata RAID such as DDF or IMSM, and one command matters most: the boot handover.

This guide follows the installed mdmon(8) from mdadm 4.3, packaged here as mdadm 4.3-1ubuntu2.1. Allow about fifteen minutes. You need a shell, an installed mdadm package, and an external-metadata MD container if you intend to test the handover. Most checks here are read-only; the handover command needs elevated privileges and should only run after the real root filesystem is mounted.

1. Check which mdmon you have

Start with ordinary, read-only checks. None of these need sudo:

$ command -v mdmon
/usr/sbin/mdmon
$ dpkg-query -W -f='${Package} ${Version}\n' mdadm
mdadm 4.3-1ubuntu2.1
$ mdmon --help
Usage: mdmon [options] CONTAINER

Options are:

Checkpoint: Record the package version and the path returned by command -v. If mdmon is missing, stop here and repair the mdadm installation through your normal package-management process.

2. Confirm this is an external-metadata setup

mdmon exists for MD arrays whose metadata is managed in userspace, such as DDF or IMSM. Native MD metadata does not normally need a separate mdmon process. Read the kernel's current summary:

$ cat /proc/mdstat
Personalities : [host-specific output]
md127 : active ...
      ...
unused devices: <none>

Look for an external: metadata version in the real output. The container itself shows a value such as external:ddf; member devices carry an external path pointing back to that container. Names and array numbers are host-specific, so do not copy md127 from this example without checking your machine.

If /proc/mdstat shows only native metadata, do not start mdmon just because the binary exists. Its manual says mdadm starts it when an external-metadata array requires it.

3. Check whether a monitor already exists

Before starting anything, inspect the current process list:

$ pgrep -af mdmon
1234 /usr/sbin/mdmon /dev/md/container

Your output may be empty, or it may list one process per active external container. An empty result is not, by itself, a repair instruction: the array may be inactive, the process may show a different path in its command line, or the boot sequence may not yet have reached the point where it starts mdmon.

The daemon polls sysfs attributes including array_state, sync_action and each member disk's state, then passes events to the metadata handler. That work matters while a filesystem uses the device, including read-only mounts, because a filesystem can still write a journal during recovery.

Checkpoint: Treat the process list and /proc/mdstat as observations. Do not kill a monitor just to make the list look tidy.

4. Understand the normal boot handover

When an external-metadata array is assembled in an initramfs, mdadm can start mdmon there. The initramfs must contain the daemon and provide a writable runtime directory for its PID and socket files; the installed manual gives /run/mdadm as the default, chosen at compile time.

After the final root filesystem has been created, usually by pivot_root, the boot sequence should run:

# mdmon --all --takeover

This is the exceptional manual invocation the manpage describes. --all finds active containers and starts a monitor for each appropriate one. --takeover replaces an existing monitor, including one started from the initramfs. Together they move monitoring to the copy installed in the real root filesystem and release the initramfs reference.

Safety warning: Run this only in the boot stage where the final root is mounted and the runtime directory is writable. Do not use --takeover as a general-purpose way to restart a healthy storage service during normal operation. If you are troubleshooting a live system, capture /proc/mdstat, the process list and boot logs first.

5. Run one container in the foreground only for diagnosis

The --foreground option stops the normal fork into the background, which is useful when a controlled boot test needs the daemon's lifetime and diagnostics attached to the invoking process:

# mdmon --foreground /dev/md/container

Replace the path with the actual container device, and only run this against a known active external-metadata container. This is not a harmless probe: it starts monitoring and stays attached. Stop the test using the service or boot supervisor that launched it, not by killing an existing production monitor.

To return to the normal arrangement, end the controlled test and let the system's mdadm boot or service workflow start the monitor again. The exact supervisor is distribution-specific, so do not invent a unit name or run a second manually managed instance.

6. Avoid member-level management mistakes

External metadata uses a container holding references to member disks and one or more sub-arrays. Some management operations belong to the container, not an individual member array; the mdmon manual specifically warns that removing or adding a disk through a member can be rejected with a message pointing you to the parent container.

That boundary is easy to miss when /proc/mdstat lists both container and member names. Before any disk-management command, identify the parent container and read the relevant mdadm(8) instructions for your metadata format. Do not use this guide to force a removal, rebuild or reshape: those operations can be destructive and sit outside a monitor handover.

Done means