Best way to store boolean values to save memory in python

Question

What is the best way to store between a million to 450,000 Boolean values in a dictionary like collection indexed by a long number? I need to use the least amount of memory pos

Answer 1

Have you thought about using a hybrid list/bitstring?

Use your list to store one dimension of your bits. Each list item would hold a bitstring of fixed length. You would use your list to focus your search to the bitstring of interest, then use the bitstring to find/modify your bit of interest.

The list should allow the most efficent recall of the bitstrings, the bitstrings should allow you to pack all your data as efficiently as possible, and the hybrid list/bitstring should allow a compromise between speed (slightly slower accessing the bit string in the list) and storage (bit packed data plus list overhead.)

Answer 2

The two main modules for this are bitarray and bitstring (I wrote the latter). Each will do what you need, but some plus and minus points for each:

bitarray

• Written as a C extension so very quick.
• Python 2 only.

bitstring

• Pure Python.
• Python 2.6+ and Python 3.x
• Richer array of methods for reading and interpreting data.

So it depends on what you need to do with your data. If it's just storage and retrieval then both will be fine, but for performance critical stuff it's better to use bitarray if you can. Take a look at the docs (bitstring, bitarray) to see which you prefer.

Answer 3

Check this question. Bitarray seems to be the preferred choice.