Which database to choose (Cassandra, MongoDB, ?) for storing and querying event / log / metrics data?

Question

In sql terms we\'re storing data like this:

table events (
  id
  timestamp
  dimension1
  dimension2
  dimension3
  etc.
)

All dimension value

Answer 1

I started to go down this path for a similar purpose (metrics gathering and reporting), and here's where I ended up...

Getting the data in is the easy part. Getting the data out is the hard part.

If you have time and talent, you could learn and use a combination of open source tools as described here: http://kibana.org/infrastructure.html. The parts list:

• Syslog-ng - Syslogd
• Logstash - Powerful log pipeline
• RabbitMQ or Redis - For queuing messages
• Elasticsearch - Full text document storage and search
• Graphite - From Orbitz, Scalable real-time graphing
• Statsd - From Etsy, counts occurrences of fields and ships to graphite
• Graphital - A ruby daemon to send host level performance data to graphite
• Kibana - A browser based log analysis front end for Logstash and Elasticsearch

If you have more money than time, consider Splunk. It's expensive, but it's a good choice for a lot of situations. e.g. I'm in a situation where the client is extremely scarce on people, but they don't mind spending money, so splunk has been a good fit in that it's more of a turn-key solution than learning and stitching together a composite of tools.