Why Systemd Services Matter

Every time your Linux server boots, dozens of services spring to life: your web server, database, SSH daemon, logging pipeline. Behind the scenes, systemd is the init system that orchestrates all of it. Understanding how to create, manage, and debug systemd services is one of the most practical skills you can develop as a Linux administrator or homelab enthusiast.

Whether you’re running a Python web app, a Docker container that should survive reboots, a custom backup script on a timer, or a Minecraft server for your friends — if it needs to run automatically and reliably, you’ll write a systemd service for it.

In this tutorial, we’ll go from zero to confident: writing your first service file, enabling it to start at boot, managing it day-to-day, and debugging when things go wrong.

Prerequisites

• A Linux system using systemd (Ubuntu 18.04+, Debian 10+, Fedora, CentOS 7+, Arch — basically any modern distro)
• SSH or terminal access with sudo privileges
• Basic familiarity with the command line
• A simple script or application to run (we’ll use a Python example)

Part 1: Anatomy of a Service File

Systemd services are defined by unit files — plain text configuration files with the .service extension. They live in:

• /etc/systemd/system/ — your custom services (this is where you’ll work)
• /lib/systemd/system/ — services installed by packages (don’t edit these directly)

Here’s a minimal service file to understand the structure:

[Unit]
Description=My Awesome Web App
After=network.target

[Service]
Type=simple
User=www-data
WorkingDirectory=/opt/myapp
ExecStart=/usr/bin/python3 /opt/myapp/server.py
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Let’s break down each section:

[Unit] — What is this service?

• Description: A human-readable name. You’ll see this in logs and status output.
• After: Tells systemd to start this service after the specified target. network.target means “wait until networking is available.” Common values: network.target, postgresql.service, docker.service.
• Requires (optional): Hard dependency — if the required unit fails, this service stops too.
• Wants (optional): Soft dependency — the other unit is started if available, but this service continues even if it fails.

[Service] — How should it run?

• Type: How systemd should track the service’s state:
  • simple (default): The process started by ExecStart is the service. Use this for most apps.
  • forking: The process forks (spawns a child) and the parent exits. Common for older daemons like nginx.
  • oneshot: The process runs once and exits. Good for scripts that do a job and finish.
  • notify: The service sends a readiness signal when it’s ready.
• User / Group: Run the service as this user. Never run services as root unless absolutely necessary.
• WorkingDirectory: The directory the service runs from. Important for apps that load relative paths.
• ExecStart: The command to start your service. Use absolute paths.
• ExecStop (optional): Custom shutdown command.
• ExecReload (optional): Command to reload configuration without full restart (e.g., /bin/kill -HUP $MAINPID).
• Restart: When should systemd restart the service?
  • no (default): Never restart.
  • on-failure: Restart on non-zero exit code. This is what you want for most services.
  • on-abnormal: Restart on signal, timeout, or watchdog.
  • always: Always restart, no matter what. Use with caution.
• RestartSec: Seconds to wait before restarting. Prevents rapid restart loops.
• Environment (optional): Set environment variables, e.g., Environment=PORT=8080 NODE_ENV=production.
• EnvironmentFile (optional): Load environment variables from a file, e.g., EnvironmentFile=/etc/myapp/env.

[Install] — When should it start?

• WantedBy: The target that “wants” this service. multi-user.target is the standard for services that should start during normal boot (think of it as “runlevel 3”). For graphical apps, use graphical.target.

Part 2: Creating Your First Service

Let’s create a real service. We’ll set up a simple Python HTTP server that serves a static directory — a common pattern for lightweight internal tools.

Step 1: Create the application

sudo mkdir -p /opt/myapp
echo '
Hello from systemd!

' | sudo tee /opt/myapp/index.html

# Create a simple Python server script
sudo tee /opt/myapp/server.py << 'EOF'
import http.server
import os
import signal
import sys

PORT = int(os.environ.get('PORT', 8080))
DIRECTORY = os.environ.get('DIRECTORY', '/opt/myapp')

def signal_handler(sig, frame):
    print('\nShutting down gracefully...')
    sys.exit(0)

signal.signal(signal.SIGTERM, signal_handler)

os.chdir(DIRECTORY)
server = http.server.HTTPServer(('0.0.0.0', PORT), http.server.SimpleHTTPRequestHandler)
print(f'Serving {DIRECTORY} on port {PORT}')
server.serve_forever()
EOF

Step 2: Create the service file

sudo tee /etc/systemd/system/myapp.service << 'EOF'
[Unit]
Description=My Simple Web App
After=network.target

[Service]
Type=simple
User=www-data
Group=www-data
WorkingDirectory=/opt/myapp
ExecStart=/usr/bin/python3 /opt/myapp/server.py
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=5
StandardOutput=journal
StandardError=journal

# Security hardening
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/opt/myapp
PrivateTmp=true

[Install]
WantedBy=multi-user.target
EOF

Notice the security hardening options at the bottom. These are optional but recommended — they restrict what the service can do even if it's compromised. We'll explain these more in a moment.

Step 3: Enable and start the service

# Tell systemd to reload its configuration (picks up the new file)
sudo systemctl daemon-reload

# Start the service now
sudo systemctl start myapp

# Enable it to start automatically on boot
sudo systemctl enable myapp

Step 4: Verify it's running

sudo systemctl status myapp

You should see something like:

● myapp.service - My Simple Web App
     Loaded: loaded (/etc/systemd/system/myapp.service; enabled; vendor preset: disabled)
     Active: active (running) since Thu 2026-06-18 12:00:00 UTC; 5s ago
   Main PID: 12345 (python3)
      Tasks: 1 (limit: 4915)
     Memory: 8.2M
        CPU: 45ms
     CGroup: /system.slice/myapp.service
             └─12345 /usr/bin/python3 /opt/myapp/server.py

Test it:

curl http://localhost:8080/

Congratulations — you just created your first systemd service!

Part 3: Day-to-Day Service Management

Here are the commands you'll use constantly:

# Start / stop / restart
sudo systemctl start myapp
sudo systemctl stop myapp
sudo systemctl restart myapp

# Reload configuration without full restart (if ExecReload is defined)
sudo systemctl reload myapp

# Reload-or-restart (tries reload first, falls back to restart)
sudo systemctl reload-or-restart myapp

# Check status
sudo systemctl status myapp

# Enable / disable auto-start on boot
sudo systemctl enable myapp
sudo systemctl disable myapp

# Enable AND start in one command
sudo systemctl enable --now myapp

# Check if a service is running (good for scripts)
systemctl is-active myapp    # returns 'active' or 'inactive'
systemctl is-enabled myapp   # returns 'enabled' or 'disabled'
systemctl is-failed myapp    # returns 'failed' or 'active'

Part 4: Reading Logs Like a Pro

Systemd captures all stdout and stderr from your service into the journal. This is one of systemd's best features — no more hunting for log files.

# View all logs for the service
sudo journalctl -u myapp

# Follow logs in real-time (like tail -f)
sudo journalctl -u myapp -f

# Show only today's logs
sudo journalctl -u myapp --since today

# Show logs from the last hour
sudo journalctl -u myapp --since "1 hour ago"

# Show logs since the last boot
sudo journalctl -u myapp -b

# Show the last 50 lines
sudo journalctl -u myapp -n 50

# Show logs with full timestamps
sudo journalctl -u myapp -o short-precise

# Show logs in JSON (for parsing)
sudo journalctl -u myapp -o json-pretty

Part 5: Debugging Failed Services

When a service doesn't start, here's your systematic approach:

Step 1: Check the status

sudo systemctl status myapp -l

The -l flag ensures long lines aren't truncated. Look for the Active line — it will say failed, inactive, or activating.

Step 2: Read the logs

sudo journalctl -u myapp -b --no-pager

This shows all logs from the current boot. The error message is usually right there.

Step 3: Common failure causes

Symptom	Likely Cause	Fix
"Executable path is not absolute"	ExecStart uses a relative path	Use full path: /usr/bin/python3 not python3
"Permission denied"	Wrong user or file permissions	Check User= and chmod/chown the files
"Address already in use"	Port conflict	Check with ss -tlnp | grep PORT
"Failed to parse service file"	Syntax error in .service file	Run systemd-analyze verify /etc/systemd/system/myapp.service
Service starts then immediately exits	App crashes or exits cleanly	Check app logs; ensure Type= is correct
"Unit not found" after editing	Forgot daemon-reload	Run sudo systemctl daemon-reload

Step 4: Test the command manually

Run the ExecStart command yourself, as the service user:

sudo -u www-data /usr/bin/python3 /opt/myapp/server.py

This reveals errors that systemd might swallow. If it works manually but not as a service, the issue is usually environment-related (missing env vars, wrong working directory, or permission issues).

Step 5: Validate the service file

sudo systemd-analyze verify /etc/systemd/system/myapp.service

This catches syntax errors and common misconfigurations before you even try to start the service.

Part 6: Security Hardening

Our example service included several security directives. Here's what they do and why they matter:

# Prevent the service from gaining new privileges (blocks setuid binaries)
NoNewPrivileges=true

# Make the filesystem read-only except for specific paths
ProtectSystem=strict

# Make /home, /root, /run/user inaccessible
ProtectHome=true

# Explicitly allow writes to specific directories
ReadWritePaths=/opt/myapp

# Give the service its own /tmp (can't see other processes' temp files)
PrivateTmp=true

# Restrict network access (if the service doesn't need network)
# PrivateNetwork=true

# Restrict system call filtering
SystemCallFilter=@system-service
SystemCallArchitectures=native

These directives create a sandbox around your service. Even if an attacker exploits a vulnerability in your application, these restrictions limit what they can do. For any service exposed to the internet, these are strongly recommended.

You can also generate a security analysis of any existing service:

systemd-analyze security myapp

This gives you a score and lists which protections are enabled. Aim for a score below 5.0 for internet-facing services.

Part 7: Running Docker Containers as Services

A very common pattern: running a Docker container as a managed service. Here's how:

sudo tee /etc/systemd/system/mycontainer.service << 'EOF'
[Unit]
Description=My Container Service
After=docker.service
Requires=docker.service

[Service]
Type=simple
User=root
ExecStartPre=-/usr/bin/docker stop mycontainer
ExecStartPre=-/usr/bin/docker rm mycontainer
ExecStart=/usr/bin/docker run \
    --name mycontainer \
    -p 8080:80 \
    -v /opt/mycontainer/data:/data \
    --restart unless-stopped \
    nginx:latest
ExecStop=/usr/bin/docker stop -t 30 mycontainer
ExecStopPost=-/usr/bin/docker rm mycontainer
Restart=on-failure
RestartSec=10
TimeoutStartSec=120

[Install]
WantedBy=multi-user.target
EOF

Key points:

• ExecStartPre with - prefix: The - means "ignore failure" — so if the container doesn't exist yet, it won't error out.
• ExecStop: Gracefully stops the container with a 30-second timeout.
• Requires=docker.service: Ensures Docker is running first.
• TimeoutStartSec: Give slow-starting containers enough time.

That said, if you're using Docker Compose, it's often simpler to just enable the Docker Compose systemd integration or use docker compose up -d with the restart: always policy. But for single containers, this pattern works great.

Part 8: Timers — Cron Jobs, But Better

Systemd can replace cron jobs with timers. They offer better logging, dependency management, and error handling.

Let's create a service that runs a backup script every day at 2 AM:

The service file (/etc/systemd/system/daily-backup.service):

[Unit]
Description=Daily Backup Script

[Service]
Type=oneshot
User=backup
ExecStart=/opt/scripts/backup.sh
Nice=19
IOSchedulingClass=idle

Note: Type=oneshot because the script runs and exits. Nice=19 and IOSchedulingClass=idle ensure the backup doesn't hog system resources.

The timer file (/etc/systemd/system/daily-backup.timer):

[Unit]
Description=Run Daily Backup at 2 AM

[Timer]
OnCalendar=*-*-* 02:00:00
Persistent=true
RandomizedDelaySec=300

[Install]
WantedBy=timers.target
EOF

Enable and start the timer (not the service!):

sudo systemctl daemon-reload
sudo systemctl enable --now daily-backup.timer

Check timer status:

systemctl list-timers --all

You'll see when the last run was and when the next run is scheduled. The Persistent=true option means if the server was off at 2 AM, the backup will run on the next boot — no missed backups.

Quick Reference Cheat Sheet

# Lifecycle
systemctl start|stop|restart|reload SERVICE
systemctl enable|disable SERVICE          # auto-start on boot
systemctl enable --now SERVICE            # enable and start
systemctl daemon-reload                   # after editing .service files

# Status & Logs
systemctl status SERVICE
systemctl is-active|is-enabled SERVICE
journalctl -u SERVICE -f                  # follow logs
journalctl -u SERVICE --since today

# Debugging
systemd-analyze verify SERVICE            # check for errors
systemd-analyze security SERVICE          # security score
systemctl list-dependencies SERVICE       # show dependency tree

# Timers
systemctl list-timers                     # show all timers
systemctl start|enable TIMER.timer

Wrapping Up

Systemd is one of those tools that seems intimidating at first but quickly becomes second nature. The key things to remember:

• Always use absolute paths in ExecStart
• Run services as a non-root user whenever possible
• Set Restart=on-failure for anything that should stay running
• Run systemctl daemon-reload after every edit to a service file
• Use journalctl to read logs — they're all there
• Add security hardening directives to any internet-facing service

Once you're comfortable with basic services, explore systemd-analyze for boot performance analysis, systemd-coredump for crash debugging, and systemd-nspawn for lightweight containers. The systemd ecosystem is deep, and every layer you learn makes your infrastructure more reliable.

Happy serving! 🐧