How systemd Unit Files Work

Introduction¶

If you run anything on a modern Red Hat, Rocky, Alma, Debian, or Ubuntu box, systemd is the thing that starts it, keeps it running, and writes down what happened when it didn't. Learning to write a unit file yourself — instead of copying one and hoping — pays off the first time a service silently fails to come back after a reboot.

By the end of this post you'll have a working service unit, understand what each section does, and know the handful of commands that tell you why a unit is in the state it's in.

Concepts¶

A unit is any object systemd manages. The common types:

Units live in three places, in ascending priority:

1. /usr/lib/systemd/system/ — shipped by packages. Don't edit these.
2. /etc/systemd/system/ — your overrides and your own units.
3. Runtime/generated units.

A drop-in file at /etc/systemd/system/foo.service.d/override.conf lets you change part of a packaged unit without replacing the whole thing.

Hands-on¶

A minimal service¶

Create /etc/systemd/system/hello.service:

[Unit]
Description=Hello world worker
# Start only after the network is actually configured.
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
ExecStart=/usr/local/bin/hello-worker
Restart=on-failure
RestartSec=5
# Run as an unprivileged user you created for this.
User=hello
Group=hello
# A few cheap hardening options.
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/hello

[Install]
WantedBy=multi-user.target

The three sections:

• [Unit] — metadata and ordering. After= controls order, not requirement; Wants=/Requires= control whether the dependency is pulled in. Wants= is the soft version — if the dependency fails, your unit still starts.
• [Service] — how to run the process. Type=simple means "the process I exec is the service". Use Type=notify if the program calls sd_notify(READY=1), Type=forking for old-style daemons that background themselves, and Type=oneshot (with RemainAfterExit=yes) for a script that runs once.
• [Install] — what happens on systemctl enable. WantedBy=multi-user.target creates a symlink so the unit starts at boot.

Load it¶

# systemd caches unit files — re-read them after any change.
sudo systemctl daemon-reload

# Start now, and start on every boot.
sudo systemctl enable --now hello.service

Timers instead of cron¶

Create hello.timer next to the service:

[Unit]
Description=Run hello-worker every 15 minutes

[Timer]
OnCalendar=*:0/15
Persistent=true

[Install]
WantedBy=timers.target

Persistent=true runs a missed job once after the machine comes back up. Enable the timer, not the service: sudo systemctl enable --now hello.timer.

Verification and troubleshooting¶

systemctl status hello.service      # state, main PID, last log lines
systemctl cat hello.service         # the effective unit incl. drop-ins
systemctl show hello.service        # every resolved property
journalctl -u hello.service -e      # full logs, jump to the end
journalctl -u hello.service --since "10 min ago"
systemctl list-timers               # next/last run of every timer

Common failure modes:

• status=203/EXEC → ExecStart path is wrong, not executable, or the interpreter (#!/usr/bin/env python3) isn't installed. Test the exact command as the User= account.
• status=200/CHDIR or status=226/NAMESPACE → a hardening directive is too strict. ProtectSystem=strict makes the whole filesystem read-only except paths in ReadWritePaths=; add the directory your service writes to.
• Unit starts, then dies, no restart → you didn't set Restart=. With Type=simple, systemd considers the service "started" the instant it execs, so a program that exits immediately looks like a clean start unless Restart=on-failure is set.
• Changes ignored → you forgot daemon-reload.
• Starts too early at boot → order it after what it needs (After=network-online.target, After=postgresql.service), and pull that dependency in with Wants=.
• enable did nothing → the unit has no [Install] section, so there's nothing for systemd to symlink.

To experiment without a file, systemd-run --unit=test --property=Type=simple /path/to/cmd creates a transient unit you can inspect and then systemctl stop test.

Related tools and reading¶

• On-site: Crontab Generator for translating an existing crontab, and the Linux commands reference.
• Related posts: systemd timers vs cron, Finding what is using a port.