Log Rotation in Node.js?

Question

In my web analytics, I am logging the data in plain text file. I want to rotate the log on a daily basis because its logging too much data. Currently I am using bunyan to ro

Answer 1

If you also want to add logrotate (e.g. remove logs that are older than a week) in addition to saving logs by date, you can add the following code:

var fs = require('fs');
var path = require("path");
var CronJob = require('cron').CronJob;
var _ = require("lodash");
var logger = require("./logger");

var job = new CronJob('00 00 00 * *', function(){
    // Runs every day
    // at 00:00:00 AM.
    fs.readdir(path.join("/var", "log", "ironbeast"), function(err, files){
        if(err){
            logger.error("error reading log files");
        } else{
            var currentTime = new Date();
            var weekFromNow = currentTime -
                (new Date().getTime() - (7 * 24 * 60 * 60 * 1000));
            _(files).forEach(function(file){
                var fileDate = file.split(".")[2]; // get the date from the file name
                if(fileDate){
                    fileDate = fileDate.replace(/-/g,"/");
                    var fileTime = new Date(fileDate);
                    if((currentTime - fileTime) > weekFromNow){
                        console.log("delete fIle",file);
                        fs.unlink(path.join("/var", "log", "ironbeast", file),
                            function (err) {
                                if (err) {
                                    logger.error(err);
                                }
                                logger.info("deleted log file: " + file);
                            });
                    }
                }
            });
        }
    });
}, function () {
    // This function is executed when the job stops
    console.log("finished logrotate");
},
true, /* Start the job right now */
'Asia/Jerusalem' /* Time zone of this job. */
);

where my logger file is:

var path = require("path");
var winston = require('winston');

var logger = new winston.Logger({
transports: [
    new winston.transports.DailyRotateFile({
        name: 'file#info',
        level: 'info',
        filename: path.join("/var", "log", "MY-APP-LOGS", "main.log"),
        datePattern: '.MM--dd-yyyy'
    }),
    new winston.transports.DailyRotateFile({
        name: 'file#error',
        level: 'error',
        filename: path.join("/var", "log", "MY-APP-LOGS", "error.log"),
        datePattern: '.MM--dd-yyyy',
        handleExceptions: true
    })
]});

module.exports = logger;

Answer 2

There's the logrotator module for log rotation that you can use regardless of the logging mechanism.

You can specify the format option to format the date format (or any other format for that matter)

var logrotate = require('logrotator');

// use the global rotator
var rotator = logrotate.rotator;

// or create a new instance
// var rotator = logrotate.create();

// check file rotation every 5 minutes, and rotate the file if its size exceeds 10 mb.
// keep only 3 rotated files and compress (gzip) them.
rotator.register('/var/log/myfile.log', {
  schedule: '5m', 
  size: '10m', 
  compress: true, 
  count: 3, 
  format: function(index) {
    var d = new Date();
    return d.getDate()+"-"+d.getMonth()+"-"+d.getFullYear();
  }
});

Answer 3

mongodb

winston itself does not support log rotation. My bad.

mongodb has a log rotation use case. Then you can export the logs to file names per your requirement.

winston also has a mongodb transport but I don't think it supports log rotation out of the box judging from its API.

This may be an overkill though.

forking bunyan

You can fork bunyan and add your repo's url in package.json.

This is the easiest solution if you're fine with freezing bunyan's feature or maintaining your own code.

As it is an open source project, you can even add your feature to it and submit a pull request to help improve bunyan.

Answer 4

Winston does support log rotation using a date in the file name. Take a look at this pull request which adds the feature and was merged four months ago. Unfortunately the documentation isn't listed on the site, but there is another pull request pending to fix that. Based on that documentation, and the tests for the log rotation features, you should be able to just add it as a new Transport to enable the log rotation functionality. Something like the following:

winston.add(winston.transports.DailyRotateFile, {
  filename: './logs/my.log',
  datePattern: '.dd-MM-yyyy'
});