logfile-analysis

I know there are other tools around like awstats or splunk, but I wonder whether there is some serious (web)server logfile analysis going on in R. I might not be the first thought to do it in R, but still R has nice visualization capabilities and also nice spatial packages. Do you know of any? Or is there a R package / code that handles the most common log file formats that one could build on? Or is it simply a very bad idea? In connection with a project to build an analytics toolbox for our Network Ops guys, i built one of these about two months ago. My employer has no problem if i open

I am going to be undertaking some logfile analyses in R (unless I can't do it in R), and I understand that my data needs to fit in RAM (unless I use some kind of fix like an interface to a keyval store, maybe?). So I am wondering how to tell ahead of time how much room my data is going to take up in RAM, and whether I will have enough. I know how much RAM I have (not a huge amount - 3GB under XP), and I know how many rows and cols my logfile will end up as and what data types the col entries ought to be (which presumably I need to check as it reads). How do I put this together into a go/nogo

I need to do some parsing of large (5-10 Gb)logfiles in Javascript/Node.js (I'm using Cube). The logline looks something like: 10:00:43.343423 I'm a friendly log message. There are 5 cats, and 7 dogs. We are in state "SUCCESS". We need to read each line, do some parsing (e.g. strip out 5 , 7 and SUCCESS ), then pump this data into Cube ( https://github.com/square/cube ) using their JS client. Firstly, what is the canonical way in Node to read in a file, line by line? It seems to be fairly common question online: http://www.quora.com/What-is-the-best-way-to-read-a-file-line-by-line-in-node-js