systemd journal: a journalctl command cheat sheet

Introduction / Why This Is Needed

journalctl is a command-line utility for viewing and managing systemd logs in Linux. It replaces traditional log files in /var/log/ and provides a unified interface for all services, the kernel, and system events. With this cheat sheet, you'll quickly master the essential commands for filtering, searching, and analyzing logs, which is critical for diagnosing failures, monitoring security, and troubleshooting service issues.

Prerequisites / Preparation

Before you begin, ensure that:

• Your distribution uses systemd (check with systemctl --version). Most modern distributions (Ubuntu 20.04+, Debian 10+, CentOS 8+, Fedora 35+) support systemd.
• You have terminal access with sudo privileges for some operations (e.g., viewing all logs or clearing them).
• The systemd-journald service is active (usually running by default). Check its status: sudo systemctl status systemd-journald.

Step 1: Basic Log Viewing

The simplest command displays all available journal entries in chronological order (oldest to newest):

journalctl

By default, the output uses a pager (usually less), allowing you to scroll through logs. To output everything at once to the terminal (without pausing), add the --no-pager flag:

journalctl --no-pager

For manual page-by-page viewing, you can pipe to less:

journalctl | less

In less, navigation: arrows to scroll, q to exit, /text to search.

Step 2: Time-Based Filtering

Often you need to see logs from a specific period. Use the --since and --until options. Time formats are flexible:

• Absolute time: "2024-01-01 14:30:00"
• Relative: "1 hour ago", "yesterday", "today"
• Combination: --since "2024-01-01" --until "now"

Example: show logs from the last 2 hours:

journalctl --since "2 hours ago"

Or for a specific day:

journalctl --since "2024-02-15" --until "2024-02-16"

Step 3: Filtering by Service (systemd Unit)

To see logs only for a specific service, use the -u (or --unit) flag with the unit name. For example, for the SSH daemon:

journalctl -u sshd.service

You can filter multiple units simultaneously:

journalctl -u nginx.service -u mysql.service

If you don't know the exact unit name, find it via systemctl list-units --type=service.

Step 4: Filtering by Priority (Log Level)

systemd journals have severity levels: emerg (critical), alert, crit, err (errors), warning, notice, info, debug. Filter by priority with the -p (or --priority) flag.

Example: only errors and higher (err, crit, alert, emerg):

journalctl -p err

Or a range: from warning to err (inclusive):

journalctl -p warning..err

The debug level will show the most detailed debugging information.

Step 5: Keyword Search

journalctl doesn't have built-in grep, but you can pipe its output to grep to search for text. This is useful for catching specific messages.

Example: find all entries containing the word "failed":

journalctl | grep "failed"

For case-insensitive search, add -i:

journalctl | grep -i "authentication"

Combine with time or service filters for precision:

journalctl -u nginx.service --since "1 hour ago" | grep "404"

Step 6: Output Limiting and Real-Time Monitoring

To avoid cluttering your terminal, limit the number of lines with the -n (or --lines) flag. It will show the last N entries:

journalctl -n 50  # last 50 lines

To follow new logs in real time (like tail -f), use -f (or --follow):

journalctl -f

Press Ctrl+C to exit follow mode.

Step 7: Output Formats and Export

By default, journalctl uses the short format (human-readable). For machine processing, other formats are available via -o (or --output):

• json or json-pretty: structured JSON.
• export: binary format for exchange.
• cat: only messages without metadata.
• short-iso: with ISO timestamps.

Example export to JSON:

journalctl -o json > logs.json

To view in cat format (message text only):

journalctl -o cat | grep "error"

Step 8: Managing Journal Size and Clearing

systemd journals can take up significant space. Check current disk usage:

journalctl --disk-usage

To clear old data, use vacuum (cleanup) commands:

• Delete logs older than N days (e.g., 7 days):

  sudo journalctl --vacuum-time=7d
  

• Limit total journal size (e.g., 100 MB):

  sudo journalctl --vacuum-size=100M
  

• Keep only the last N journal files:

  sudo journalctl --vacuum-files=5
  

⚠️ Important: Clearing permanently deletes data. Ensure you don't need old logs for auditing or investigation.

Step 9: Working with Boots

The systemd journal automatically segments logs by system boot. View a list of available boots with timestamps:

journalctl --list-boots

The output shows the boot index (0 = current, -1 = previous, etc.) and time ranges.

To view logs for a specific boot, use -b (or --boot) with the index:

journalctl -b -1  # logs from the previous boot

Step 10: Advanced Combinations and Examples

Combine filters for precise searching. Here are useful examples:

• Logs for a specific service from the last hour:

  journalctl -u nginx.service --since "1 hour ago"
  

• Kernel (kern) errors at level err and higher:

  journalctl -p err -k
  

• Logs from the current boot, sorted by time (reverse order):

  journalctl -b -r  # -r for reverse (newest to oldest)
  

• Search logs with highlighting (using less and grep):

  journalctl | grep --color -E "error|failed|warning"
  

Verifying Results

After running commands, ensure that:

• The output contains expected entries (e.g., when filtering by service, you see only that service's logs).
• After clearing, journalctl --disk-usage shows reduced size.
• Exported files (JSON, TXT) open correctly and contain data.

If the output is empty, check:

• Correctness of filters (time, unit name).
• Availability of logs for the specified period (journalctl --list-boots).
• Access permissions (you might need sudo).

Common Issues

1. "Permission denied" error or empty output without sudo

Some logs (especially system ones) are only accessible by root. Solution: use sudo before the command or configure access groups (e.g., add the user to the systemd-journal group).

2. Too much data, command "hangs"

The journal can be huge. Always apply filters (-u, --since, -p) or limits (-n). For a quick view of recent entries: journalctl -n 100 --no-pager.

3. journalctl doesn't find logs for a certain period

The system may have been rebooted, and logs from that period are stored in a different "boot". Use journalctl --list-boots to find the correct index, then journalctl -b <index>.

4. Cleanup issues: "No space left on device" after vacuum

The --vacuum command might not delete files if they are in use. Stop critical services before cleaning or use --vacuum-time cautiously. As a last resort, temporarily stop systemd-journald: sudo systemctl stop systemd-journald, delete files in /var/log/journal/, then restart the service.

5. journalctl command missing

In rare cases (very old distributions or minimal installs), systemd might not be installed. Check presence: which journalctl. If missing, install systemd via your package manager (e.g., sudo apt install systemd on Debian/Ubuntu), but be cautious: replacing the init system can disrupt operations.