Logback: SizeAndTimeBasedRollingPolicy applies totalSizeCap to each day in maxHistory

Question

Logback version 1.2.3

I want to use a SizeAndTimeBasedRollingPolicy in our logback configuration file (logback.xml), but at this time the SizeAndTimeBasedRollingPolicy doesn't function as expected. (https://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy)

Ideally I want to keep logs dating back no later than ex. 90 days, each file no larger than 100MB and a total archive size of ex. 10GB in total.

As it currently stands, the totalSizeCap is applied to each entry within the MaxHistory range. Ex.

<appender name="ROLLING" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>mylog.txt</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
      <!-- rollover daily -->
      <fileNamePattern>mylog-%d{yyyy-MM-dd}.%i.txt</fileNamePattern>
       <!-- each file should be at most 100MB, keep 60 days worth of history, but at most 20GB -->
       <maxFileSize>100MB</maxFileSize>    
       <maxHistory>60</maxHistory>
       <totalSizeCap>1GB</totalSizeCap>
    </rollingPolicy>
    <encoder>
      <pattern>%msg%n</pattern>
    </encoder>
</appender>

The above XML configuration will create logs spanning over 60 days, applying a totalSizeCap of 1GB per day. This will lead to a total archive size of 60GB instead of the expected 1GB. If the totalSizeCap is reached during the day the day's logs will start to rollover by deleting the day's oldest files, this will create gaps within the log history, which we don't want. A work-around for this error was to use a yearly rollover, instead of daily or monthly rollover, unfortunately yearly rollover doesn't work when using a SizeAndTimeBasedRollingPolicy.

Does anyone know of this problem, has this been fixed, or am I doing something wrong in my configuration?

Answer 1

It seems that you have found a bug!

There should be no gaps in the logs. When totalSizeCap is reached, the oldest log file should be deleted. When maxHistory reached, the oldest log file should be deleted.

Unfortunately it seems that there is a bug in logback which causes gaps in the logging because not the oldest files are deleted. See demonstration here: https://github.com/riskop/slf4j_logback_SizeAndTimeBasedRollingPolicy_problem

I've opened an issue: https://jira.qos.ch/browse/LOGBACK-1361

According to Gülcü it will be fixed in logback classic 1.3.0.

Note that you can "vote" the issue on logback's Jira!