Archive Linux Crash Records Safely with systemd-pstore

systemd-pstore.service quietly deletes the kernel crash records it archives, so it pays to know its defaults before you actually need one. This walkthrough checks whether the service can run on this machine, shows where it copies pstore records, and helps you set a storage policy on purpose. The examples match systemd 255.4 on Ubuntu 24.04, and nothing here manufactures a crash record or forces a service run against live pstore data.

Allow about fifteen minutes. You need a shell and, for configuration changes or service control, an account that can use sudo. Most inspection commands are unprivileged. Keep a maintenance window available before starting the service manually: processing normally removes records from the small pstore filesystem after archiving them.

1. Check the installed service

Start with read-only checks. They confirm the unit and show whether its start conditions are currently satisfied:

$ systemctl --version
systemd 255 (255.4-1ubuntu8.17)
$ systemctl is-enabled systemd-pstore.service
enabled
$ systemctl status systemd-pstore.service --no-pager

The exact status output varies. On this host the service is enabled but inactive because its start condition is unmet. The unit is ordered before sysinit.target, runs once, and is skipped unless /sys/fs/pstore is non-empty and the machine is not a container.

Checkpoint: inspect the unit without starting it:

$ systemctl cat systemd-pstore.service
$ findmnt /sys/fs/pstore
$ ls -la /sys/fs/pstore

A mounted but empty directory is a normal state. It means there are no records for the service to archive now, not that the service has failed.

2. Understand the archive destinations

When the service starts, the systemd-pstore executable reads pstore.conf and processes files in /sys/fs/pstore/. The configured Storage= value controls the copy destination:

• external is the default. It saves files under /var/lib/systemd/pstore/ and writes them to the journal.
• journal writes the contents to the journal only.
• none exits without processing pstore files.

Unlink=true is also the default. After a file has been archived, the service removes it from pstore so the limited backend has room for a later kernel error. That removal is the normal protection against the store filling up, but it means pstore is not a second copy of the archive.

Read the effective local configuration before changing anything:

$ sed -n '1,200p' /etc/systemd/pstore.conf
$ find /etc/systemd/pstore.conf.d -maxdepth 1 -type f -name '*.conf' -print 2>/dev/null

The main file and snippets use the [PStore] section. Snippets in /etc/systemd/pstore.conf.d/ take precedence over vendor configuration, and later filenames win when a single-value option is repeated.

3. Choose a policy before testing

For a normal host, leave Storage=external and Unlink=true unless your retention design says otherwise. The external directory gives you files to copy into your incident archive, while the journal provides the usual query path.

If a separate retention system already captures the journal and you do not want local archive files, use a drop-in for journal-only storage. This is a persistent configuration change and needs elevated privileges:

$ sudo install -d -m 0755 /etc/systemd/pstore.conf.d
$ sudo sh -c 'printf "%s\n" "[PStore]" "Storage=journal" > /etc/systemd/pstore.conf.d/90-local.conf'

Do not set Storage=none as a troubleshooting shortcut. It disables processing and leaves records in the backend. Likewise, setting Unlink=false can preserve records for inspection but can eventually prevent a later crash from being recorded because pstore space is commonly small.

Undo the example by removing the drop-in, then inspect the directory again:

$ sudo rm /etc/systemd/pstore.conf.d/90-local.conf
$ ls -l /etc/systemd/pstore.conf.d

The removal is safe only if that file is the one you created for this guide. Do not delete a directory or a vendor file as a substitute. The executable reads the configuration when it starts, so a future service start uses the restored settings.

4. Review records without opening a service change

If pstore contains records, first decide whether they are already covered by your incident process. Listing names is read-only:

$ sudo find /sys/fs/pstore -maxdepth 1 -type f -printf '%f\n'

Do not edit, move or remove those files by hand while investigating a kernel failure. Their names and contents can be useful evidence. The service's archive operation is the controlled hand-off, and its default unlink behaviour means that starting it is not a read-only test.

Past service messages and archived contents in the journal can be queried without restarting anything:

$ journalctl -u systemd-pstore.service -b --no-pager
$ journalctl -b --no-pager | grep -i pstore

The second command is a broad search and may include unrelated kernel messages. Treat the journal as evidence, not as proof that every record was archived. Check the configured destination and the service's own messages together.

5. Start the service only when you accept the hand-off

When that decision is made, start the oneshot service with elevated privileges:

$ sudo systemctl start systemd-pstore.service
$ systemctl status systemd-pstore.service --no-pager
$ sudo find /sys/fs/pstore -maxdepth 1 -type f -printf '%f\n'
$ journalctl -u systemd-pstore.service -b --no-pager

An empty pstore directory after a successful run is expected with the default Unlink=true. With Storage=external, also check the archive directory:

$ sudo find /var/lib/systemd/pstore -maxdepth 1 -type f -printf '%f\n' 2>/dev/null

This unit has RemainAfterExit=yes, so a successful start normally leaves it active with an exited process until it is stopped. An unmet condition leaves it inactive. In both cases, use the start result, journal messages and destination contents as the evidence that matters.

6. Diagnose a skipped or empty run

If systemd says the start condition was unmet, check the two conditions in the unit: whether /sys/fs/pstore is non-empty and whether the system is a container. A container normally cannot provide the host's persistent crash backend, so investigate this on the host rather than trying to override the condition blindly.

If records remain after a run, inspect Storage=, Unlink=, permissions and the service journal. A journal-only policy does not create files under /var/lib/systemd/pstore/. A non-empty pstore directory after processing may be intentional when Unlink=false, or may indicate that archiving failed. Do not repeatedly start the service as a repair loop: preserve the failure message and fix the destination or retention problem first.

The service is independent of kdump. pstore is a small, local post-mortem path that can survive a reboot; it is not a replacement for a tested crash-dump design or a remote incident archive.