mongoDB 32-bit 2GB limit, best practices

Question

Mongo newbie running 32-bit Mongo 2.0.1 on Windows XP. No option at present to run 64-bit.

I want to use Mongo to map-reduce a set of 60 files, each a monthly snaps

Answer 1

If you don't have 64-bit, my best recommendation is you don't. It would be too cumbersome to manage them to be under 32-bit process. It would probably be better to explore hadoop for your purpose.

Answer 2

You can go over the 2Gb limit by sharding. The 2Gb limit applies to individual mongod processes, rather than the total data in the sharded dataset.

Here is some documentation about getting started with sharding. There is also a Python script for setting up a sharded environment on a single machine. I assume it will work on Windows.