Write a systemd.service Unit That Fails Loudly

A service that silently pretends to start is worse than one that fails loudly, and Type=exec in a systemd.service unit gets you the loud kind. You will write one, validate it without starting anything, enable it, and check its process, logs and restart behaviour. Allow 15 to 20 minutes.

• You need: a program that can run continuously, its absolute path, and administrative access for files under /etc/systemd/system and for systemctl operations.
• Targets: systemd 255, package version 255.4-1ubuntu8.17 here. Replace example-worker and /usr/local/bin/example-worker with your real values; the sample assumes the program stays in the foreground and takes no special arguments.

1. Confirm the executable before writing the unit

Run these checks as your ordinary user, not with sudo. A service can fail simply because its path is wrong, the file is not executable, or it depends on a working directory that does not exist.

$ test -x /usr/local/bin/example-worker && echo executable
executable
$ /usr/local/bin/example-worker --version
example-worker 1.2.3

If the first command prints nothing, fix the installation or use the correct absolute path. systemd does not search your interactive shell's aliases or functions, and relying on a user-specific PATH makes a unit harder to audit.

2. Write a foreground service unit

Create /etc/systemd/system/example-worker.service:

[Unit]
Description=Example worker
After=network-online.target
Wants=network-online.target

[Service]
Type=exec
ExecStart=/usr/local/bin/example-worker
Restart=on-failure
RestartSec=5s
TimeoutStartSec=30s
TimeoutStopSec=30s

[Install]
WantedBy=multi-user.target

• Type=exec is the honest default for a normal long-running program on systemd 255: it reports start failure if the binary cannot be executed or basic process setup fails. Type=simple can report success before the binary has actually run.
• Keep the program in the foreground. A daemon that forks itself needs a different design; Type=forking is discouraged wherever notify, notify-reload or a foreground process is possible.
• Restart=on-failure covers an unexpected non-zero exit, signal termination and relevant timeouts, but it will not restart the service after a deliberate systemctl stop. The five-second delay guards against a tight restart loop.

3. Validate before starting

Ask systemd to parse the file. This does not start the service.

$ sudo systemd-analyze verify /etc/systemd/system/example-worker.service
$

No output and exit status 0 means the unit passed this syntax and dependency check, though warnings are still worth reading. Common traps: a relative executable path, shell syntax such as pipes in ExecStart, or multiple commands on one line. ExecStart is not run by a shell; if you need one, name it explicitly and account for the extra quoting and error handling.

4. Load, start and inspect the service

$ sudo systemctl daemon-reload
$ sudo systemctl enable --now example-worker.service
Created symlink /etc/systemd/system/multi-user.target.wants/example-worker.service -> /etc/systemd/system/example-worker.service
$ systemctl is-enabled example-worker.service
enabled
$ systemctl is-active example-worker.service
active

The exact symlink message can vary, and an already-enabled unit may produce no creation message. The two final words are what actually matter. For fuller evidence, check the unit and recent journal entries.

$ systemctl status --no-pager example-worker.service
$ journalctl -u example-worker.service -n 30 --no-pager

Read the status output for the main PID, the loaded unit path and the last result. If it is not active, run systemctl status and journalctl before changing anything: the journal usually separates an executable error, a permissions problem and an application-level failure.

5. Test failure and recovery safely

A controlled stop is a useful undo path in its own right.

$ sudo systemctl stop example-worker.service
$ systemctl is-active example-worker.service
inactive
$ sudo systemctl start example-worker.service
$ systemctl is-active example-worker.service
active

A stop does not remove the unit or its boot enablement. To undo the installation completely, stop it, disable it, then remove the unit file and reload systemd.

$ sudo systemctl disable --now example-worker.service
$ sudo rm /etc/systemd/system/example-worker.service
$ sudo systemctl daemon-reload

6. Know when the simple pattern is insufficient

• Type=oneshot for a task that should run and finish, normally with RemainAfterExit=yes if it should still count as logically active afterwards. It is not a general daemon replacement.
• Type=notify only when the program sends systemd a READY=1 notification; systemd waits for that signal before considering startup complete.
• Type=notify-reload when the program implements the matching reload protocol and should receive SIGHUP for reloads.
• Do not use ExecStartPre for a long-running helper. systemd kills processes left over from that preparatory command before the main service starts; use it for short checks or setup instead.
• Use ExecStopPost for cleanup that must run even after a failed startup.
• If you add ExecStop, make it wait for the program to finish. Merely sending an asynchronous signal can still lead to immediate service cleanup.