Logfile analysis in R?

Question

I know there are other tools around like awstats or splunk, but I wonder whether there is some serious (web)server logfile analysis going on in R. I might not be the first thoug

Answer 1

I have used R to load and parse IIS Log files with some success here is my code.

Load IIS Log files
require(data.table)

setwd("Log File Directory")

# get a list of all the log files
log_files <- Sys.glob("*.log")

# This line
# 1) reads each log file
# 2) concatenates them
IIS <- do.call( "rbind", lapply( log_files,  read.csv, sep = " ", header = FALSE, comment.char = "#", na.strings = "-" ) )

# Add field names - Copy the "Fields" line from one of the log files :header line 
colnames(IIS) <- c("date", "time", "s_ip", "cs_method", "cs_uri_stem", "cs_uri_query", "s_port", "cs_username", "c_ip", "cs_User_Agent", "sc_status", "sc_substatus", "sc_win32_status", "sc_bytes", "cs_bytes", "time-taken")

#Change it to a data.table
IIS <- data.table( IIS )

#Query at will
IIS[, .N, by = list(sc_status,cs_username, cs_uri_stem,sc_win32_status) ]