Web server log rotation setup

A quiet-sounding problem can take a site down: a web server writes an access and an error log for every request, and on a busy site those files grow on the order of hundreds of megabytes a day. Left alone they fill the disk, and a web server that cannot write its logs can stop serving traffic. Log rotation is not polishing; it is the thing that keeps the log files small enough to debug later and small enough not to kill the box.

Why logs grow

Nginx and Apache write access and error logs by appending. The classic sign is disk full errors or a 502/500 storm with no obvious code cause, and the check is obvious once you look: a multi-gigabyte access.log. The same trapped growth applies to PHP-FPM and CMS logs. Rotation replaces the growing file with a scheduled, dated family of smaller files, compresses old ones, and deletes the oldest after a retention window.

Set up rotation

The standard tool is logrotate, driven by per-log rules under /etc/logrotate.d/. A Nginx rule that rotates daily, keeps four weeks of history, and compresses older files looks like:

/var/log/nginx/*.log {
    daily
    missingok
    rotate 28
    compress
    delaycompress
    dateext
    create 0640 nginx adm
    sharedscripts
    postrotate
        test -r /var/run/nginx.pid && kill -USR1 `cat /var/run/nginx.pid`
    endscript
}

The copytruncate alternative truncates the live log rather than renaming it. The postrotate with kill -USR1 tells Nginx to reopen its log file after the rename, which matters because a web server keeps the old inode open and would otherwise keep writing to the renamed file. The sharedscripts and the PID check keep one reopen command across all matching logs.

Rotate in place

The two commands that everyone trips over are the reopen and the chopping off of a still-open file. A web server holds the log file descriptor open for its lifetime. If rotation renames access.log to access.log.1 while Nginx is still running, Nginx keeps writing into the old file under its new name unless you send it the reopen signal. On Nginx the signal is USR1; on Apache you restart or use graceful. The copytruncate option avoids this by copying and truncating instead of renaming, at the cost of a larger write.

Match retention to need

Rotate daily and keep enough for the debugging and compliance window you actually use. Four weeks of daily files is a sane default; longer retention means more disk. Because rotation only reorganizes what is in the log, the total disk used is bounded by the retention window times the daily rate. Right-size retention so the maximum footprint fits comfortably on the disk even during a traffic surge.

Verify and monitor

After configuring, run logrotate --debug to see what would happen and logrotate -f once to force a rotation and check the result. Confirm the new .1/.gz files appear and the live log starts from zero. Add the log directory to disk monitoring so that a rotation failure, for instance a timer that never ran or a config parse error, shows up as a warning before the disk fills.

Related server health

Disk-full failures and log rotation are two sides of the same operational concern. The disk-full guide covers the other writers that can exhaust a filesystem, and the error-log guide covers how to turn the rotated files into a diagnosis rather than just storage.