Node.js monitor file for changes and parse them

问题

I need to monitor a file for changes. Due to a large amount of new entries to this file I would need to 'monitor' this file. I would need to get the new inserted content to this file to be able to parse this content.

I found this code:

fs.watchFile('var/log/query.log', function() {
console.log('File Changed ...');
//how to get the new line which is now inserted?
});

回答1:

On bash you would do something like that with tail --follow.

There is also a package tail availible. you can watch on a file, and get new lines with an event:

const Tail = require('tail').Tail;
var tail = new Tail("var/log/query.log");
tail.watch()
tail.on("line", data => {
  console.log(data);
});

回答2:

Here is an example of how I used fs.watchFile to monitor a log file for a game called Hearthstone and pick up new log entries to monitor game events as they happened while playing. https://github.com/chevex-archived/hearthstone-log-watcher/blob/master/index.js

var fs = require('fs');
var options = {
  logFile: '~/Library/Preferences/Blizzard/Hearthstone/log.config',
  endOfLineChar: require('os').EOL
};
// Obtain the initial size of the log file before we begin watching it.
var fileSize = fs.statSync(options.logFile).size;
fs.watchFile(options.logFile, function (current, previous) {
  // Check if file modified time is less than last time.
  // If so, nothing changed so don't bother parsing.
  if (current.mtime <= previous.mtime) { return; }

  // We're only going to read the portion of the file that
  // we have not read so far. Obtain new file size.
  var newFileSize = fs.statSync(options.logFile).size;
  // Calculate size difference.
  var sizeDiff = newFileSize - fileSize;
  // If less than zero then Hearthstone truncated its log file
  // since we last read it in order to save space.
  // Set fileSize to zero and set the size difference to the current
  // size of the file.
  if (sizeDiff < 0) {
    fileSize = 0;
    sizeDiff = newFileSize;
  }
  // Create a buffer to hold only the data we intend to read.
  var buffer = new Buffer(sizeDiff);
  // Obtain reference to the file's descriptor.
  var fileDescriptor = fs.openSync(options.logFile, 'r');
  // Synchronously read from the file starting from where we read
  // to last time and store data in our buffer.
  fs.readSync(fileDescriptor, buffer, 0, sizeDiff, fileSize);
  fs.closeSync(fileDescriptor); // close the file
  // Set old file size to the new size for next read.
  fileSize = newFileSize;

  // Parse the line(s) in the buffer.
  parseBuffer(buffer);
});

function stop () {
  fs.unwatchFile(options.logFile);
};

function parseBuffer (buffer) {
  // Iterate over each line in the buffer.
  buffer.toString().split(options.endOfLineChar).forEach(function (line) {
    // Do stuff with the line :)
  });
};

It first calculates the initial size of the file because in this log watcher module I only want to read new data as it's being written by the game. I don't care about existing data. It then starts watching the file for changes. When the change handler fires we check if the modified time is really newer because some other changes about the file can trigger the handler when no data we care about actually changed. We wanted this watcher to be as performant as we could.

We then read the new size of the file and calculate the difference from the last time. This tells us exactly how much data to read from the file to get only the newly written data. Then we store the data in a buffer and parse it as a string. Just split the string by newline characters. Using core module os to get os.EOL will give you the correct line ending character for the operating system you are running on (windows line ending character is different from linux/unix).

Now you have an array of lines written to the file :)

来源：https://stackoverflow.com/questions/38190773/node-js-monitor-file-for-changes-and-parse-them

标签

javascript

node.js

watch

logfile