systemd Service Failed to Start: A Diagnostic Walkthrough


A service will not come up:

Job for myapp.service failed because the control process exited with error code.
See "systemctl status myapp.service" and "journalctl -xeu myapp.service" for details.

systemd is telling you where the answer is, and the answer is genuinely there. The status output gives you an exit code that narrows the cause immediately, and the journal usually contains the underlying error verbatim. This is a five-minute problem if you read them in the right order.

Key Takeaways

  • systemctl status gives an exit code that narrows the cause before you read a single log line.
  • journalctl -eu SERVICE shows the service’s own output, which usually names the real error.
  • Exit code 203 means the executable path is wrong or not executable; 200 and 209 point at user or permission settings.
  • Always run systemd-analyze verify after editing a unit file to catch syntax errors.
  • Use systemctl edit to override packaged units so upgrades do not discard your changes.

Quick Fix

Read the service’s own log output rather than the generic failure message. The journal almost always contains the underlying error — a missing file, a bad config line, or a permission denial — stated plainly.

journalctl -eu myapp.service --no-pager -n 50

Step 1: Read the Status Output

systemctl status myapp.service

Two lines matter. The Active: line tells you the pattern, and the Process: line gives you the exit code.

The pattern tells you a lot on its own:

  • failed (Result: exit-code) — the program ran and returned an error. The journal will say why.
  • activating (auto-restart) — it is crash-looping. Something fails immediately at startup.
  • inactive (dead) — it exited without error, usually because the service type is wrong.

And systemd’s exit codes are specific enough to be diagnostic:

  • 203/EXEC — systemd could not execute the binary. Wrong path, missing file, or not executable.
  • 200/CHDIR — the WorkingDirectory does not exist.
  • 209/STDOUT or 208/STDERR — output redirection failed, often a log path the user cannot write.
  • 217/USER — the User= account does not exist.
  • 1 — the application itself failed. Go straight to the journal.

Step 2: Read the Journal

journalctl -eu myapp.service --no-pager -n 50

The -u filters to this unit and -e jumps to the end. This is where the real error lives — a config file it could not parse, a port already taken, a database it could not reach.

If the service is crash-looping, watch it live in one terminal while restarting it in another:

journalctl -fu myapp.service

To see everything since the last boot, which helps when a service fails only at startup:

journalctl -u myapp.service -b --no-pager

Cause 1: Wrong Path (Exit 203)

Check what the unit is trying to run:

systemctl cat myapp.service | grep ExecStart

Then verify that file exists and is executable:

ls -l /usr/local/bin/myapp

Two rules catch most of these. ExecStart requires an absolute path — python app.py fails, /usr/bin/python3 /opt/app/app.py works. And the file needs the executable bit:

sudo chmod +x /usr/local/bin/myapp

Note also that systemd does not run a shell, so pipes, redirects, and environment variable expansion in ExecStart do not work. If you need them, wrap the command:

ExecStart=/bin/bash -c '/usr/local/bin/myapp >> /var/log/myapp.log 2>&1'

Cause 2: Permissions

If the unit specifies User=, that account must exist and be able to reach everything the service touches. Confirm the user exists:

id myappuser

Then check the paths it needs — the working directory, log files, sockets, data directories:

sudo -u myappuser test -r /opt/myapp/config.yml && echo readable || echo DENIED

Running the command as that user is the fastest way to reproduce a permission failure outside systemd:

sudo -u myappuser /usr/local/bin/myapp

If it fails here with a clear error, you have found the problem without systemd in the way.

Cause 3: The Wrong Service Type

A service that reports success but is not running usually has a Type= mismatch. Type=simple expects the process to stay in the foreground. If your app daemonises itself, systemd sees the parent exit and considers the service finished.

Either run the app in the foreground — most have a flag for this — or tell systemd what to expect:

[Service]
Type=forking
PIDFile=/run/myapp.pid

For a one-shot task that legitimately exits, use Type=oneshot with RemainAfterExit=yes so its completed state is recorded.

Cause 4: Dependencies and Ordering

A service that fails at boot but starts fine manually is starting too early — before the network, or before a database it needs. Make the ordering explicit:

[Unit]
After=network-online.target postgresql.service
Wants=network-online.target

After= controls ordering; Wants= or Requires= controls whether the dependency is pulled in at all. Ordering alone does not start anything.

For services that must wait for something systemd does not track, a short delay is a pragmatic fallback:

[Service]
ExecStartPre=/bin/sleep 5

After Any Edit

Always validate before restarting, and always reload systemd so it re-reads the file:

sudo systemd-analyze verify /etc/systemd/system/myapp.service
sudo systemctl daemon-reload
sudo systemctl restart myapp.service

Forgetting daemon-reload is a common source of “I fixed it but nothing changed” — systemd is still running the old definition.

For packaged services, never edit the file in /lib/systemd/system/; a package upgrade overwrites it. Create an override instead:

sudo systemctl edit nginx

Verify the Fix

systemctl is-active myapp.service && systemctl is-enabled myapp.service

You want active and enabled — the second confirms it will also come back after a reboot, which is the failure people discover weeks later. If you can afford it, reboot and confirm, since boot-time ordering problems only appear at boot.

Prevent It From Recurring

Give long-running services a restart policy so a transient failure recovers itself:

[Service]
Restart=on-failure
RestartSec=5

Keep unit files in version control alongside the application they run, and run systemd-analyze verify in CI so a malformed unit never reaches a server. When you inherit a failing service, systemctl cat shows the effective configuration including every override — start there rather than hunting through directories.

Frequently Asked Questions

What does exit code 203 mean?

systemd could not execute the program in ExecStart. The path is wrong, the file does not exist, or it is not marked executable. ExecStart also requires an absolute path — a bare command name will fail this way.

Why does my service work manually but fail under systemd?

systemd runs it with a different environment, user, and working directory, and without a shell. Environment variables from your profile are absent. Set Environment=, WorkingDirectory=, and User= explicitly in the unit.

Why did my change to the unit file do nothing?

systemd caches unit definitions. Run sudo systemctl daemon-reload after any edit, then restart the service. Without the reload, systemd keeps using the previously loaded version.

How do I stop a service from restarting in a loop?

Run sudo systemctl stop SERVICE, then read journalctl -eu SERVICE to find the startup error. Temporarily setting Restart=no in an override makes the failure easier to inspect without the loop interfering.

Scroll to Top