A simple log file format

Question

I\'m not sure if it was asked, but I couldn\'t find anything like this.

My program uses a simple .txt file for log purposes, It just creates/opens a file and appends

Answer 1

The 3 W's :

When, what and where.

For viewing something like multitail ("tail on steroids") http://www.vanheusden.com/multitail/

or for pure ms windows try mtail http://ophilipp.free.fr/op_tail.htm

And to keep your files readable, you might want to start new files when if the filesize of the current log file is over certain limit. Example:

• activity0.log (1 mb)
• activity1.log (1 mb)
• activity2.log (1 mb)
• activity3.log (1 mb)
• activity4.log (205 bytes)

Answer 2

First, check you're only logging things that are useful.

If it's all useful, make sure it is easily parsable by tools such as grep, that way you can find the info you want. Make sure you have the type of log entry, the date/time all conforming to a layout.

Build yourself a few scripts to extract the information for you. Alternatively, use separate log files for different types of entries.

Answer 3

If only the size of the log file is the problem, I recommend using logrotate or something similar. logrotate watches log files and, depending on how you configured it, after a given time or when the log file exceeds a given size, it moves the log file to an archive directory and optionally compresses it. Then the original log file is truncated. For example, you could configure it to archive the log file every 24 hours or whenever the files size exceeds 500kb.

Answer 4

LogExpert

I've found it here. Filter is better, than in mtail. There's an option of highlighting just adding a string and the app is nice and readable. You can customize columns as you like.

Answer 5

A fairly standard way to deal with logging from an application into a plain text file is to:

• split the logs into different program functional areas.
• rotate the logs on a daily/weekly basis (i.e. split the log on a size or date basis)
  • so the current log is "mylog.log" or whatever, and yesterday's was "mylog.log.1" or "mylog.ddmmyyyy.log"

This keeps the size of the active log manageable. And then you can just have expiry rules so that old logs get thrown away on a regular basis.

In addition it would be a good idea to have different log levels for your application (info/warning/error/fatal) so that you're not logging more than is necessary.

Answer 6

Basically you better just split logs according to severity. You'll rarely need to read all logs for the whole system. For example apache allows to configure error log and access log, pretty obvious what info exactly they have.

If you're under linux system grep is your best tool to search through logs for specific entries.