Software for Webserver Log Analysis? [closed]

Answer 1

Splunk > is great, and free. It allows you to visualize and search all of your logs in real time. And it's all web based, so you can view your logs from just about anywhere.

Answer 2

My old company always used WebLog Expert. There is a free 'lite' version. Its still has on-going development and can be combined with geo-location databases if you use one of the paid versions.

Answer 3

I get Awstats and Webalizer with my web hosting account and I find that neither is accurate or very useful. The reported numbers are inflated by up to 1000%, because the tools don't properly identify bots and spiders. Here is a comparison of the Visits metric between three tools over the past 3 weeks (I think Awstats has only partial data for 3/23, and no data for today 3/24, which is why I did not include the numbers).

                            Google  Woopra  Awstats
Sunday, March 1, 2009           10      11      69
Monday, March 2, 2009           13      14      85
Tuesday, March 3, 2009          13      14      96
Wednesday, March 4, 2009        21      28      91
Thursday, March 5, 2009         19      25     107
Friday, March 6, 2009           12      10      88
Saturday, March 7, 2009         12      14     100
Sunday, March 8, 2009           10      11      65
Monday, March 9, 2009           13      14      78
Tuesday, March 10, 2009         17      13      96
Wednesday, March 11, 2009       18      16      87
Thursday, March 12, 2009        19      18      87
Friday, March 13, 2009          12      13      66
Saturday, March 14, 2009        11       7      52
Sunday, March 15, 2009          11      12      57
Monday, March 16, 2009          13      15      92
Tuesday, March 17, 2009         24      22     102
Wednesday, March 18, 2009       18      16      79
Thursday, March 19, 2009        17      18      73
Friday, March 20, 2009          16      11      70
Saturday, March 21, 2009        24      26      67
Sunday, March 22, 2009         103     114     216
Monday, March 23, 2009         232     223     117

I personally prefer Woopra over Google. While it is still in Beta, it can take a long time for your site to get approved, and it will probably be a paid service at some point, the real-time monitoring capabilities are amazing. The new custom reporting capabilities on Google Analytics are superior to Woopra, though. Woopra does not have any capabilties to produce printed reports

Answer 4

I use Analog because it's free, but it is rather dated now (wow, last release was 4 years ago!) and I'm sure it doesn't have as many fancy features as newer ones.

Answer 5

It seems that as javascript page-tagging becomes the more popular choice as a way of processing web stats, there's not as much work being done on log-based analysis tools anymore in the marketplace. My office used to use a product called LiveStats.XSP. It wasn't the greatest tool by any means, but it did have some nice features. It was recently bought by Microsoft and is no longer supported however. It abandoned log analysis turned into a proposed Google Analytics killer called Microsoft Gatineau, which supposedly is good at determining the demographics of your visitors, including age and gender (yeah sure...)

When I was looking for log analysis software a while ago, I wanted to avoid anything that looked overly bloated and enterprisey, which is what most stuff seemed to be, focusing more on the marketing and advertising aspects of reports.

One thing you may want to look at is the new version of Urchin, Urchin6 (see features here). Urchin I believe was bought by Google a few years ago. It's offered as a locally installed solution, and with it you have the option to use either page-tagging or log file analysis for any site that it monitors. There also seems to be some interface ties between Google's own web-based Google Analytics and Urchin. It's not free though, unfortunately, and I think you can only get it through authorized partners.

It does all the standard logfile analysis stuff, everything is browser-based, the reports it offers are pretty deep and comprehensive, and it also seems to have a few bells and whistles that other services don't offer. For example, I remember it being able to present a view of a web page it tracks with colored hot spots overlayed on top of it, based on how often users click on items on that area of the page. Worth checking out the demo of it anyways.

Answer 6

AWStats and Webalizer are both good and free (I think both free speech as well as free beer). I generally prefer the look of AWStats - it has a nice modern look whereas Webalizer looks like something created in about 1992.

They both give roughly the same information which includes:

• Most frequently accessed pages
• Which hosts (IPs and Domain Names) visitors come from
• Proportion of users using different browsers
• Proportion of downloads of different file types

All of this information is usually viewable on a hour by hour, day by day, month by month and year by year basis. Normally the raw data is available but also with bar charts and pie charts. Both AWStats and Webalizer will (I think) try and work out where your visitors come from by using services such as GeoIP, although I never bothered to set this up. Some also try to work out what order people have visited pages in and things like that - but that is very difficult to do so the results are guesses at the best.

I generally find them both useful - even if just to get an overview of what is going on with my server and who is accessing it. They are both relatively easy to install - although I seem to remember Webalizer being a little easier than AWStats, and they both have varied configuration options to let you decide exactly what you want to get out of them.

For more information see their sites at awstats.sourceforge.net/ and http://www.webalizer.org/.

Hope that helps.

Robin