How to configure log4j to only keep log files for the last seven days?

Question

I have the following logging problem with several Java applications using log4j for logging:

I want log files to be rotated daily, like

Answer 1

My script based on @dogbane's answer

/etc/cron.daily/hbase

#!/bin/sh
find /var/log/hbase -type f -name "phoenix-hbase-server.log.[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]" -exec bzip2 {} ";"
find /var/log/hbase -type f -regex ".*.out.[0-9][0-9]?" -exec bzip2 {} ";"
find /var/log/hbase -type f -mtime +7 -name "*.bz2" -exec rm -f {} ";"

/etc/cron.daily/tomcat

#!/bin/sh
find /opt/tomcat/log/ -type f -mtime +1 -name "*.[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9].*log" -exec bzip2 {} ";"
find /opt/tomcat/log/ -type f -mtime +1 -name "*.[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9].txt" -exec bzip2 {} ";"
find /opt/tomcat/log/ -type f -mtime +7 -name "*.bz2" -exec rm -f {} ";"

because Tomcat rotate needs one day delay.

Answer 2

There is another option DailyRollingFileAppender. but it lacks the auto delete (keep 7 days log) feature you looking for

sample

log4j.appender.DRF=org.apache.log4j.DailyRollingFileAppender
log4j.appender.DRF.File=example.log
log4j.appender.DRF.DatePattern='.'yyyy-MM-dd

I do come across something call org.apache.log4j.CompositeRollingAppender, which is combine both the features of the RollingFileAppender (maxSizeRollBackups, no. of backup file) and DailyRollingFileAppender (roll by day).

But have not tried that out, seems is not the standard 1.2 branch log4j feature.

Answer 3

I create this Methode and call it by closing the application:

  public void deleteFiles(){

    File f = new File("log");
    File[] fileArray = f.listFiles();
    double timenow = System.currentTimeMillis();

    double olderTenDays = timenow - 864000000;// MS for ten days

    for (int i = 0; i < fileArray.length; i++) {

        if(fileArray[i].lastModified()< olderTenDays )
           fileArray[i].delete();
    }
 }

Answer 4

Use the setting log4j.appender.FILE.RollingPolicy.FileNamePattern, e.g. log4j.appender.FILE.RollingPolicy.FileNamePattern=F:/logs/filename.log.%d{dd}.gz for keeping logs one month before rolling over.

I didn't wait for one month to check but I tried with mm (i.e. minutes) and confirmed it overwrites, so I am assuming it will work for all patterns.

Answer 5

If you are using Linux, you can configure a cron job using tmpwatch.

Most Linux systems have a tmpwatch cron job that cleans up the /tmp directory. You can add another that monitors your logging directory and deletes files over 7 days old.

If you are using a different system, there are probably equivalent utilities.

Answer 6

I had set:

log4j.appender.R=org.apache.log4j.DailyRollingFileAppender
log4j.appender.R.DatePattern='.'yyyy-MM-dd
# Archive log files (Keep one year of daily files)
log4j.appender.R.MaxBackupIndex=367

Like others before me, the DEBUG option showed me the error:

log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.

Here is an idea I have not tried yet, suppose I set the DatePattern such that the files overwrite each other after the required time period. To retain a year's worth I could try setting:

log4j.appender.R.DatePattern='.'MM-dd

Would it work or would it cause an error ? Like that it will take a year to find out, I could try:

log4j.appender.R.DatePattern='.'dd

but it will still take a month to find out.