Configuring Log Rotation in Linux: A Step-by-Step Guide

Introduction / Why This Is Needed

Logging is a cornerstone of diagnostics in Linux. However, without control, log files can grow indefinitely, filling the /var partition and causing service failures. Log rotation is the automated process of archiving, compressing, and deleting outdated records. Using the standard logrotate utility solves the problem with minimal effort, saving space and maintaining a manageable event history. After completing this guide, you will have a working rotation system for both system logs and your application logs.

Requirements / Preparation

Before you begin, ensure that:

1. You have access to an account with sudo privileges.
2. Your distribution is based on Debian/Ubuntu or RHEL/CentOS/Fedora (the configuration is universal, but paths may differ slightly).
3. The logrotate utility is installed (usually present by default). You can verify with:

   logrotate --version
   

4. You know the paths to the log files that need rotation (e.g., /var/log/nginx/access.log).

Step-by-Step Guide

Step 1: Review the Default Configuration

The main configuration file is /etc/logrotate.conf. Open it in an editor (sudo nano /etc/logrotate.conf). Typical contents look like this:

# see "man logrotate" for details
# rotate log files weekly
weekly

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones
create

# use date as a suffix of the rotated file
#dateext

# uncomment this if you want your log files compressed
compress

# RPM packages drop log rotation information into this directory
include /etc/logrotate.d

Key parameters:

• weekly — rotation frequency (daily, weekly, monthly, yearly).
• rotate 4 — keep 4 archived files; older ones will be deleted.
• compress — compress rotated logs using gzip (files will get a .gz extension).
• create — create a new empty log file after rotation.
• include /etc/logrotate.d — include additional configurations from this directory.

These are global default settings. For specific logs, they can be overridden.

Step 2: Create a Configuration for Your Application

Best practice is to create separate configs in /etc/logrotate.d/. For example, for the Nginx web server, which writes logs to /var/log/nginx/, or for a custom application myapp.

1. Create the configuration file:

   sudo nano /etc/logrotate.d/myapp
   

2. Add a rule. Below is an example for application logs in /var/log/myapp/:

   /var/log/myapp/*.log {
       daily
       missingok
       rotate 14
       compress
       delaycompress
       notifempty
       create 644 myappuser myappgroup
       sharedscripts
       postrotate
           systemctl reload myapp.service > /dev/null 2>&1 || true
       endscript
   }
   

   Block breakdown:
   • /var/log/myapp/*.log — path to files to rotate (glob patterns are supported).
   • daily — rotate every day.
   • missingok — do not throw an error if the log file is missing.
   • rotate 14 — keep 14 archives (approximately two weeks).
   • delaycompress — compress not immediately, but on the next rotation (so processes holding the file can close it).
   • notifempty — do not rotate empty files.
   • create 644 myappuser myappgroup — after rotation, create a new file with specified permissions and owner.
   • sharedscripts — run postrotate/prerotate scripts only once, not for each file.
   • postrotate ... endscript — command executed after rotation. Here we gracefully reload the myapp service so it starts writing to the new file. This is critical for many daemons!

💡 Tip: For logs that are actively written to (e.g., access.log), always use postrotate with a service reload/signal (kill -USR1 $(cat /var/run/nginx.pid)). For static files, scripts are not needed.

Step 3: Syntax Check and Test Run

Do not wait for the next daily cron run. Check the configuration:

1. Dry run (debug mode):

   sudo logrotate -d /etc/logrotate.conf
   

   
   The -d flag enables debug mode. You will see which files will be rotated and what actions are planned, but no changes will be applied. Look in the output for lines like running postrotate script and paths to your logs.
2. Force run: If the dry run succeeds, perform an actual rotation:

   sudo logrotate -f /etc/logrotate.conf
   

   
   The -f (force) flag makes logrotate perform rotation regardless of the time since the last run.
3. Verify the result:
   • In the logs directory (/var/log/myapp/), compressed archives like myapp.log.1.gz (last rotation) and myapp.log.2.gz etc. should appear.
   • The current active log (myapp.log) should be newly created (if you used create).
   • Check file modification times: ls -laht /var/log/myapp/.

Step 4: Scheduling (cron)

By default, logrotate is run from system cron daily via a script in /etc/cron.daily/logrotate. You can change the frequency by creating a symlink in /etc/cron.hourly/, /etc/cron.weekly/, or by adding your own task to /etc/crontab. However, daily is usually sufficient.

⚠️ Important: Do not run logrotate too frequently (e.g., every hour) for logs with low intensity. This can lead to rotation of empty files and accumulation of many small archives.

Step 5: Monitoring and Debugging

If rotation is not working:

1. Check file permissions for the log files and the /etc/logrotate.d/ directory. The root user must have access.
2. Ensure there are no syntax errors in the config (curly braces must be balanced).
3. Check if a program is locking the file. Sometimes the copytruncate option helps (copies the file and truncates the original, but can cause record loss during copying). Use it only if postrotate is unavailable.
4. Check logrotate's own logs. They are typically written to syslog (/var/log/syslog or /var/log/messages). Look for entries with the logrotate tag.

Verification

After completing steps 2-3:

1. Ensure that archived files with a .gz extension have appeared in the target log directory.
2. The size of the current log file (myapp.log) should be close to zero (if the application continues writing) or equal to the size of records that appeared since the last rotation.
3. The number of rotated files (.log.1.gz, .log.2.gz...) does not exceed the rotate N value.
4. The application continues to write correctly to the new log file (check if new entries are added).

Common Issues

• Error "error: error opening /var/log/myapp/*.log: No such file or directory"
  • Cause: The log file does not exist at rotation time.
  • Solution: Add the missingok parameter to the configuration block. This is safe if the log is created dynamically by the application.
• Logs continue to grow indefinitely; rotation does not occur
  • Cause: logrotate is not running via cron, or the configuration file has incorrect permissions/name.
  • Solution: Check that the file in /etc/logrotate.d/ has a .conf extension (not .txt). Run sudo logrotate -f /etc/logrotate.conf manually and check the output.
• Application stops writing to log after rotation
  • Cause: The application's file descriptor remains open on the old (deleted) file.
  • Solution: Must use a postrotate block with a command that forces the application to reopen the log file (e.g., systemctl reload, kill -USR1, or a similar command for your software). Avoid copytruncate for important logs.
• No disk space after rotation (due to compression)
  • Cause: The compression process (gzip) creates a temporary copy of the file, requiring free space equal to the size of the original log.
  • Solution: Ensure there is enough free space on the /var partition (at least the size of the largest log). As a last resort, temporarily disable compress in the config.
• Wrong files are being rotated
  • Cause: Incorrect glob pattern in the path.
  • Solution: Test the pattern with ls /var/log/myapp/*.log. Ensure it matches the intended files and does not capture extra ones.

Advanced Settings (Optional)

• size 100M — rotate not based on time, but when the file reaches the specified size. Can be combined with daily (rotation occurs if the file is large or the day has come).
• maxsize 500M — rotate if the file is larger than this size, even if the period has not arrived (e.g., daily + maxsize).
• olddir /var/log/myapp/archived — move rotated files to the specified subdirectory (must exist).
• mail root — email rotated logs (requires a configured MTA).
• su root myappuser — perform rotation as a different user (useful if logrotate runs as root but logs belong to another user).

These parameters are added to the configuration block (e.g., /etc/logrotate.d/myapp).