Setting Up logrotate in Linux: A Step-by-Step Guide to Log Rotation

Introduction / Why This Is Needed

Log files (/var/log/) are a key source of information for diagnosing problems in a Linux system. Over time, they can consume gigabytes of disk space, slow down applications, and make it difficult to find relevant entries. Logrotate is the standard utility that automatically manages logs: it rotates (renames old logs, creates new ones), compresses archives, and deletes outdated files. In this guide, you will configure logrotate for your specific tasks to:

• Free up disk space by automatically removing old logs.
• Simplify analysis by breaking large files into manageable parts (daily, weekly).
• Preserve important data in a compressed format (*.gz).
• Set a retention policy separately for each application.

Prerequisites / Preparation

Before you begin, ensure that:

1. You have sudo privileges or access to the root user. Configuring system logs requires elevated privileges.
2. The logrotate utility is installed. It is included in the base installation of virtually all distributions (Ubuntu/Debian, CentOS/RHEL, Fedora). Verify with: logrotate --version.
3. You understand your application's log structure: where they are located (/var/log/...), and how they are named (app.log, error.log).
4. You know your rotation policy: how often you want to rotate (daily, weekly), how many archives to keep, and whether compression is needed.

Step-by-Step Guide

Step 1: Understand the Logrotate Configuration Structure

The main config file is /etc/logrotate.conf. It looks roughly like this:

# see "man logrotate" for details
# rotate log files weekly
weekly

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones
create

# use date as a suffix of the rotated file
#dateext

# uncomment this if you want your log files compressed
#compress

# RPM packages drop log rotation information into this directory
include /etc/logrotate.d

Key points:

• The directives at the top (weekly, rotate 4) are global default settings.
• The line include /etc/logrotate.d pulls in per-application configurations from that directory. This is where we will add our custom rules.
• Commented-out parameters (#dateext, #compress) can be uncommented to enable them.

Step 2: Create a Configuration for Your Application

Assume you have an application that writes logs to /var/log/myapp/access.log and /var/log/myapp/error.log.

1. Create the configuration file:

   sudo nano /etc/logrotate.d/myapp
   

2. Add the following content (detailed explanation below):

   /var/log/myapp/*.log {
       daily
       rotate 14
       compress
       delaycompress
       missingok
       notifempty
       create 644 root root
       sharedscripts
       postrotate
           systemctl reload myapp.service > /dev/null 2>&1 || true
       endscript
   }
   

   
   Block breakdown:
   • /var/log/myapp/*.log — path to log files (wildcards are supported).
   • daily — rotate every day. Alternatives: daily, weekly, monthly, yearly.
   • rotate 14 — keep 14 most recent archives. Older ones will be deleted.
   • compress — compress rotated logs using gzip (files become *.log.1.gz).
   • delaycompress — do not compress the most recent archive (*.log.1), but start from the second one (*.log.2.gz). This allows the process writing to *.log.1 to continue uninterrupted.
   • missingok — do not throw an error if the log file is missing.
   • notifempty — do not rotate an empty file.
   • create 644 root root — after rotation, create a new empty log file with the specified permissions and ownership.
   • sharedscripts — run postrotate/prerotate scripts once for the entire block, not for each file.
   • postrotate ... endscript — command executed after rotation. Here we reload the application's service (systemctl reload) so it starts writing to the new file. This is critical for many daemons! For simple files written via >> (appending), you may not need postrotate. The || true phrase suppresses an error if the service is not running.

Step 3: Configure Parameters for System Logs (Optional)

Standard system logs (e.g., syslog, auth.log) already have configurations in /etc/logrotate.d/. You can modify them. For example, in /etc/logrotate.d/syslog:

/var/log/syslog
/var/log/auth.log
/var/log/kern.log
/var/log/dmesg {
    rotate 7
    daily
    missingok
    notifempty
    delaycompress
    compress
    postrotate
        /usr/lib/rsyslog/rsyslog-rotate
    endscript
    sharedscripts
    create 640 root adm
    mode 640
    owner root
    group adm
}

Note: syslog often has a postrotate to restart rsyslog.

Step 4: Test Your Configuration

NEVER test on "live" important logs without a backup!

1. Dry Run (Debug Mode):

   sudo logrotate -d /etc/logrotate.conf
   

   
   The -d flag enables debug mode. You will see which files would be rotated and what commands would be executed, but no changes are applied. Look for errors (error:) in the output.
2. Force Rotation for a Test File: Create a test log:

   echo "Test log entry $(date)" | sudo tee /var/log/myapp/test.log
   

   
   Force rotation for the specific config:

   sudo logrotate -f /etc/logrotate.d/myapp
   

   
   The -f (force) flag makes logrotate rotate the file even if the time/size criteria aren't met. Check the result:

   ls -lh /var/log/myapp/
   

   
   You should see test.log (new, empty) and test.log.1 (old, possibly compressed).
3. Check Status: Logrotate stores information about its last run in /var/lib/logrotate/status. Ensure the date updated:

   cat /var/lib/logrotate/status
   

Step 5: Ensure It Runs Automatically via Cron

Logrotate is typically run from cron once a day. Check:

cat /etc/crontab | grep logrotate

A typical line:

17 5 * * * root test -x /usr/sbin/logrotate && /usr/sbin/logrotate /etc/logrotate.conf

This means the utility runs daily at 5:17 AM. You can change the schedule by editing this file or adding a job to /etc/cron.daily/logrotate (if your distribution uses that).

Verification

1. Wait a day or run cron manually: sudo run-parts /etc/cron.daily.
2. Check that old logs were renamed and compressed:

   sudo find /var/log/myapp -type f -name "*.log*" -exec ls -lh {} \;
   

3. Ensure new logging continues in the fresh access.log file (add a test entry to your app and verify).
4. Confirm old archives are deleted after exceeding the rotate limit.

Troubleshooting

⚠️ Error: "error: error opening /var/log/...: Permission denied"Cause: The user running logrotate (usually root) lacks read/write permissions on the log files or directory. Solution: Ensure permissions on log files and their directory allow root read/write. For example: sudo chown root:root /var/log/myapp && sudo chmod 755 /var/log/myapp.

⚠️ Logs are not rotating even though time has passedCause 1: Size threshold (if size is used) not reached, or period (daily/weekly) not elapsed. Solution 1: Use logrotate -f to force rotation or wait for the schedule. For testing, you can temporarily add size 1k. Cause 2: The config has notifempty and the log file is empty. Solution 2: Remove notifempty or ensure the log receives entries.

⚠️ Application continues writing to the old (rotated) fileCause: The postrotate script to reload/signal the application failed. Solution: Manually test the command in postrotate. Ensure the service name (systemctl reload myapp) is correct and the service exists. For applications that don't handle SIGHUP, you may need restart or a full stop/start.

💡 Tip: Rotating logs written via copytruncate If your application cannot be reloaded (doesn't support SIGHUP), use the copytruncate option:

/var/log/myapp/*.log {
    daily
    rotate 7
    compress
    copytruncate
    missingok
    notifempty
}

How it works: logrotate makes a copy of the log file and then truncates the original file to zero length. This is safe but carries a risk of losing entries written during the copy. Use only if there is no alternative.

💡 Tip: Rotation by size To rotate a log when it reaches a specific size (e.g., 100 MB), add size 100M:

/var/log/myapp/*.log {
    size 100M
    rotate 5
    compress
    delaycompress
    missingok
    notifempty
}

Now rotation will occur as soon as the file exceeds 100 MB, even if less than a day has passed since the last rotation.