Handling Huge Multidimensional Arrays in C++

Question

I\'m designing a game in C++ similar to Minecraft that holds an enormous amount of terrain data in memory. In general, I want to store an array in memory that is [5][4][5][50][

Answer 1

Here's something that works and can be built upon without the boost dependency. One downside is it removes use of [][][] style of referencing elements, but it's a small cost and can be added.

template<class T>
class Matrix {
    unsigned char* _data;
    const size_t _depth;
    const size_t _cols;
    const size_t _rows;
public:
    Matrix(const size_t& depth, const size_t& rows, const size_t& cols):
        _depth(depth),
        _rows(rows), 
        _cols(cols) {
        _data = new unsigned char [depth * rows * cols * sizeof(T)];
    }
    ~Matrix() {
        delete[] _data;
    }
    T& at(const size_t& depthIndex, const size_t& rowIndex, const size_t& colIndex) const {
        return *reinterpret_cast<T*>(_data + ((((depthIndex * _cols + colIndex) * _rows) + rowIndex) * sizeof(T)));
    }
    const size_t& getDepth() const {
        return _depth;
    }
    const size_t& getRows() const {
        return _rows;
    }
    const size_t& getCols() const {
        return _cols;
    }
};

int _tmain(int argc, _TCHAR* argv[])
{
    Matrix<int> block(50, 50, 50);
    size_t d, r, c;
    for (d = 0; d < block.getDepth(); d++) {
        for (r = 0; r < block.getRows(); r++) {
            for (c = 0; c < block.getCols(); c++) {
                block.at(d, r, c) = d * 10000000 + r * 10000 + c;
            }
        }
    }
    for (d = 0; d < block.getDepth(); d++) {
        for (r = 0; r < block.getRows(); r++) {
            for (c = 0; c < block.getCols(); c++) {
                assert(block.at(d, r, c) == d * 10000000 + r * 10000 + c);
            }
        }
    }
return 0;
}

Answer 2

I did this by putting all the data in a binary file. I calculated the offset of the data and used seek() and read() to get the data when needed. The open() call is very slow so you should leave the file open during the lifetime of the program.