Can OLAP be done in BigTable?

Question

In the past I used to build WebAnalytics using OLAP cubes running on MySQL. Now an OLAP cube the way I used it is simply a large table (ok, it was stored a bit smarter than that

Answer 1

If you are looking for a table-scan approach, have you considered Google BigQuery? BigQuery does automatic scale-out on the back-side that gives interactive response. There is a good session by Jordan Tigani from the 2012 Google I/O event that explains some of the internals.

http://www.youtube.com/watch?v=QI8623HlYd4

It's not MapReduce but it is geared towards high-speed table scan like what you described.

Answer 2

It's even kind of been done (kind of).

LastFm's aggregation/summary engine: http://github.com/zohmg/zohmg

A google search turned up a google code project "mroll" but it doesn't have anything except contact info (no code, nothing). Still, might want to reach out to that guy and see what's up. http://code.google.com/p/mroll/

Answer 3

My answer relates to HBase, but applies equally to BigTable.

Urban Airship open-sourced datacube, which I think is close to what you want. See their presentation here.

Adobe also has a couple of presentations (here and here) on how they do "low-latency OLAP" with HBase.

Answer 4

Andrei Dragomir made an interesting talk about how Adobe performs OLAP functionality with M/R and HBase.

Video: http://www.youtube.com/watch?v=5U3EnfiKs44

Slides: http://hstack.org/hbasecon-low-latency-olap-with-hbase/

Answer 5

We managed to create low latency OLAP in HBase by preagragating a SQL query and mapping it into appropriate Hbase qualifiers. For more detail visit below site.

http://soumyajitswain.blogspot.in/2012/10/hbase-low-latency-olap.html