Why Python `Memory Error` with list `append()` lots of RAM left

Question

I am building a large data dictionary from a set of text files. As I read in the lines and process them, I append(dataline) to a list.

At some point th

Answer 1

If you're using a 32-bit build of Python, you might want to try a 64-bit version.

It is possible for a process to address at most 4GB of RAM using 32-bit addresses, but typically (depending on the OS), one gets much less. It sounds like your Python process may be hitting this limit. 64-bit addressing removes this limitation.

edit Since you're asking about Windows, the following page is of relevance: Memory Limits for Windows Releases. As you can see, the limit per 32-bit process is 2, 3 or 4GB depending on the OS version and configuration.