log4net, can 2 applications write to the same log file?

Question

Is it possible to have 2 applications write to the same log file using log4net?

Answer 1

They can but if one application is writing the the file the other application will most likely experience an error if it needs to write to the log as well due to the fact that the first application will be holding the file open for writing. It is always best to have dedicated logging sources for your applications - if you need to share a log, use a database as it is designed to handle concurrent writes.

This is one of those things that will work really well on your machine when you are developing since you are not likely to create enough concurrent writes to the log file to notice any problems. Once your application begins to experience more load the problem will begin to show itself and at that point it may manifest itself in strange ways. I would definitely try another solution.

Answer 2

Yes, it is possible, as stated above, but I've now done some stress testing of this scenario.

The setup is quite simple:

1. Web project 1 set up with a page that logs a single entry + a button that sets off 1000 requests to the same page, with a counter in the URL (picked up by the logging statement).
2. Web project 2 set up identically, against the same log file.

When both buttons are clicked at the same time, the log entries are interspersed throughout the log. BUT, and this is the big GOTCHA, judging by the accompanying request counter, it's clear that there are race conditions. Almost every time one web project succeeds in logging its entry, the other one fails (the entry is skipped).

So, with decent traffic against this common log, you'll basically have no guarantee of which log statements actually end up in the log. The conclusion is to always have project specific log files, it seems.

The test was done with the default of "MinimalLock". I redid the test with "ExclusiveLock" as well, only to find that the first web project to configure the logger "won", basically locking out ALL other requests to log. So obviously, that's a no-go as well.

Answer 3

It depends on the FileAppender's LockingModel. If it is ExclusiveLock then another process cannot open the file for writing. The alternative is MinimalLock, but it is not meant for this purpose. It is intended for allowing another process to move or delete the file.

Answer 4

Or you can use Mutex to lock on common resource and therefore to synchronize access to a common log file from different processes.

Answer 5

MinimalLock partially solves the problem (as @Mark mentioned), but if you're using RollingFileAppender, you'll run into other problems. When the file rolls, you may find yourself in a race condition where one process overwrites another process's newly created log file.

Other options include RemoteLogger, where you have a simple server set up to receive and record logging events sent by other processes. Likewise, you can log to a SQL database. I wrote a simple appended that logs to Redis; you'd need a simple application to read from Redis and record to a file. The problem with these approaches is they all introduce a point of failure. When something isn't working right is often when you need logs the most, and then they might not be available.

So my solution was to avoid the problem altogether by having each process log to its own file. This was easy to do with a change in configuration. In your (Rolling)FileAppender configuration, use:

<file type="log4net.Util.PatternString" value="c:\mylog-[%processid].txt" />

The process ID becomes part of the file name. Yes, this means you now have several log files to comb through, but a log file aggregator like Graylog, Splunk, or Logscape can help.