What happens if I log into the same file from multiple different processes in python?

Question

I spent hours to dig the behavior, first about those questions:

• Atomicity of `write(2)` to a local filesystem
• How can I synchronize -- make atomic -- writ

Answer 1

I would not rely on tests here. Weird things can only happen in race conditions, and exhibiting a race condition by test is almost non sense because the race condition is unlikely to occur. So it can work nicely for 1000 test runs and randomly break later in prod... The page you cite says:

logging to a single file from multiple processes is not supported, because there is no standard way to serialize access to a single file across multiple processes in Python

That does not means that it will break... it could even be safe in a particular implementation on a particular file system. It just means that it can break without any hope of fix on any other version of Python or on any other filesystem.

If you really want to make sure of it, you will have to dive into Python source code (for your version) to control how logging is actually implemented, and control whether is it safe on your file system. And you will always be threatened by the possibility that a later optimization in logging module breaks your assumptions.

IMHO that is the reason for the warning in Logging Cookbook, and the existence of a special module to allow concurrent logging to the same file. This last one does not rely on anything unspecified, but just uses explicit locking.