Minimal overhead distributed event logging library/framework?

Question

We\'d like to keep records of all major events in our systems. For example, where the database might store the current user status, the event log should record all changes to th

Answer 1

If you want to go down the UDP route (as you seem happy with that), and Java is an option, then check out Log4j and its support for UDP transmission via the Log4j UDPAppender.

LoggingEvent will take a java.lang.Object as a message, so it's pretty generic and you can throw whatever data you want into that. If you're going across the network it should (most likely) be serialisable, and given that you want UDP, should be of a comensurate size - 64k or less, and then dependent on the transport layer). You'll simply have to intercept the LoggingEvent on the server side and then process it however you want.

Note that the UDP appender comes as a companion component to Log4j and you'll have to build it yourself. But that's trivial.

Answer 2

You may consider using old good *nix Syslog. It has very small overhead and is mostly used over UDP or local UNIX sockets, but may use TCP if you need reliable logging. Works for my (Python/Perl, mostly, but it is completely language/platform-agnostic) like a charm.

Sorry, I'm not familiar with Java, but feature-wise, this seems to be some good library I've googed: http://syslog4j.org/

Edit: Quick googling discovered an article called "Robust event logging with Syslog", which seems to be pretty detailed on the subject. Sorry, I've misread it when posted and thought it is a *nix syslog library, but it isn't.

Answer 3

It sounds like a potential candidate for messaging (fire and forget). I'm a .NET person mostly so don't know what logging frameworks there are out there for Java. But I had a quick look to see if there are any messaging appenders for log4j (I use log4net quite often)- IBM have an article on a WebSphere MQ JMS appender, which might be helpful to you.

So rather than take my answer as advocating the use of WebSphere MQ- please take as a suggestion to consider messaging- there are lots of open source messaging frameworks out there- RabbitMQ is just one example.