Systemd Services: Complete Guide to Managing Linux Services

Introduction / Why This Matters

Systemd is the modern init system and service manager for most Linux distributions (Ubuntu, Debian, CentOS, Fedora). Understanding systemd services is critical for server administration, automation setup, and troubleshooting.

After completing this guide, you will be able to:

• Manage built-in and custom services via systemctl
• Read and analyze logs with journalctl
• Create, enable, and disable services
• Diagnose common startup errors

Requirements / Preparation

1. Operating System: Linux with systemd (verify: pidof systemd should return a PID).
2. Access Rights: Root privileges are required to manage system services (use sudo).
3. Basic Knowledge: Ability to work with the terminal, edit files (nano/vim), and understand Linux directory structure.
4. Recommended Utilities: systemctl, journalctl, cat, less, grep (all are pre-installed).

Essential Service Management Commands

Systemctl is the primary tool for interacting with systemd. Here are the key operations:

Viewing Service Lists

# All active services
systemctl list-units --type=service --state=running

# All services (including inactive ones)
systemctl list-unit-files --type=service

# Filter by name
systemctl list-units --type=service | grep nginx

Managing Service State

# Start/stop/restart
sudo systemctl start service_name.service
sudo systemctl stop service_name.service
sudo systemctl restart service_name.service

# Reload configuration without stopping (reread)
sudo systemctl reload service_name.service

# Check status
systemctl status service_name.service

💡 Tip: You can omit the .service extension—systemd adds it automatically: systemctl status nginx instead of nginx.service.

Viewing Service Logs (journalctl)

Systemd stores logs in a binary journal accessible via journalctl.

Common Usage Scenarios

# Logs for a specific service
sudo journalctl -u nginx.service

# Logs from the last hour
sudo journalctl -u nginx.service --since "1 hour ago"

# Real-time tracking (like tail -f)
sudo journalctl -u nginx.service -f

# Filter by priority (err, warning, info)
sudo journalctl -u nginx.service -p err

# Logs since the last reboot
sudo journalctl -u nginx.service -b

# Clean up the journal (if it's using too much space)
sudo journalctl --vacuum-time=3d  # keep logs for 3 days

Troubleshooting

# Find errors in logs from all services
sudo journalctl -p err --since today

# Show the last 50 lines of nginx logs
sudo journalctl -u nginx -n 50

# Export logs to a file
sudo journalctl -u nginx > nginx_logs.txt

Creating Your Own systemd Service

Let's create a service to run a custom script.

Step 1: Prepare the Script

Create the script to be executed. Example: /opt/myscript.sh

#!/bin/bash
# Example script that writes to a log every minute
LOG_FILE="/var/log/myscript.log"
echo "$(date): myscript started" >> $LOG_FILE

Make it executable:

sudo chmod +x /opt/myscript.sh

Step 2: Create the Unit File

Create the file /etc/systemd/system/myscript.service:

[Unit]
Description=My Custom Service
After=network.target

[Service]
Type=simple
ExecStart=/opt/myscript.sh
Restart=on-failure
RestartSec=10
User=root
Group=root

[Install]
WantedBy=multi-user.target

Parameter Explanation:

• After=network.target — start after the network is up.
• Type=simple (default) — the process starts directly.
• Restart=on-failure — restart if the service crashes.
• RestartSec=10 — wait 10 seconds before restarting.
• WantedBy=multi-user.target — the target for autostart (multi-user mode).

Step 3: Activate the Service

# Reload systemd configuration
sudo systemctl daemon-reload

# Start now
sudo systemctl start myscript.service

# Enable at boot
sudo systemctl enable myscript.service

# Check status
systemctl status myscript.service

Step 4: Verify Operation

# Service logs
sudo journalctl -u myscript.service -f

# File created by the script
sudo cat /var/log/myscript.log

Managing Autostart and Dependencies

systemd Targets

Analogous to SysVinit runlevels:

• multi-user.target — multi-user mode (no GUI).
• graphical.target — with graphical interface.
• rescue.target — emergency mode.

Check the current default target:

systemctl get-default

Change the target (e.g., switch to graphical mode):

sudo systemctl set-default graphical.target

Service Dependencies

In the [Unit] section, you can specify:

Requires=nginx.service    # Requires nginx to be running
After=nginx.service       # Start after nginx
Wants=nginx.service       # Wants nginx to start (optional)

Example for a web app that depends on a database:

[Unit]
Description=My Web App
Requires=postgresql.service
After=postgresql.service

Verification Checklist

1. Service Status: systemctl status service_name should show active (running).
2. Autostart: systemctl is-enabled service_name → enabled.
3. Logs: journalctl -u service_name shows no Failed or timeout errors.
4. Process: ps aux | grep script_name shows the running process.
5. Functionality: If the service provides a network service (web server, DB), test its availability (e.g., curl http://localhost).

Common Issues

1. Service Fails to Start Silently with systemctl start

• Cause: Error in the unit file or script.
• Solution: Check journalctl -u service -b and systemctl status service. Common errors:
  • Incorrect path in ExecStart (use absolute paths).
  • Script not executable (chmod +x).
  • Port already in use (for network services).

2. Service Starts Then Immediately Crashes

• Cause: Script exits with an error.
• Solution: Add Type=oneshot + RemainAfterExit=yes for one-time tasks, or ensure the script doesn't exit (daemonize it). Check environment variables—systemd may not provide the expected PATH.

3. Permission Error When Creating Unit File

• Cause: File not owned by root or has wrong permissions.
• Solution:

  sudo chown root:root /etc/systemd/system/*.service
  sudo chmod 644 /etc/systemd/system/*.service
  

4. journalctl Logs Empty for a Service

• Cause: Service doesn't write to stdout/stderr, or uses syslog instead of journald.
• Solution: Ensure the script outputs to the console (echo, print). If using an external logger (e.g., nginx writes to /var/log/nginx), check its logs directly.

5. Dependencies Not Working

• Cause: Invalid target in WantedBy/Requires.
• Solution: Verify the target exists: systemctl list-units --type=target. Use multi-user.target for background services, graphical.target for GUI apps.

6. Service Not Enabled at Boot

• Cause: Missing [Install] section with WantedBy.
• Solution: Add to the unit file:

  [Install]
  WantedBy=multi-user.target
  

  Then run sudo systemctl daemon-reload && sudo systemctl enable service_name.

Additional Resources

• Man Pages: man systemd, man systemctl, man systemd.unit, man journalctl.
• Official Documentation: https://www.freedesktop.org/wiki/Software/systemd/
• Diagnostics: sudo systemd-analyze blame — see which services take long to start.
• Editing Existing Services: /lib/systemd/system/ (package-managed) vs /etc/systemd/system/ (local overrides). Only modify files in /etc/ to prevent package updates from overwriting your changes.

Systemd is a powerful tool. Experiment with test services, study the logs, and gradually increase configuration complexity. Good luck with your administration!