Very large matrices using Python and NumPy

Question

NumPy is an extremely useful library, and from using it I\'ve found that it\'s capable of handling matrices which are quite large (10000 x 10000) easily, but begins to strug

Answer 1

Are you asking how to handle a 2,500,000,000 element matrix without terabytes of RAM?

The way to handle 2 billion items without 8 billion bytes of RAM is by not keeping the matrix in memory.

That means much more sophisticated algorithms to fetch it from the file system in pieces.