'and' (boolean) vs '&' (bitwise) - Why difference in behavior with lists vs numpy arrays?

Question

What explains the difference in behavior of boolean and bitwise operations on lists vs NumPy arrays?

I\'m confused about the appropriate use of

Answer 1

and tests whether both expressions are logically True while & (when used with True/False values) tests if both are True.

In Python, empty built-in objects are typically treated as logically False while non-empty built-ins are logically True. This facilitates the common use case where you want to do something if a list is empty and something else if the list is not. Note that this means that the list [False] is logically True:

>>> if [False]:
...    print 'True'
...
True

So in Example 1, the first list is non-empty and therefore logically True, so the truth value of the and is the same as that of the second list. (In our case, the second list is non-empty and therefore logically True, but identifying that would require an unnecessary step of calculation.)

For example 2, lists cannot meaningfully be combined in a bitwise fashion because they can contain arbitrary unlike elements. Things that can be combined bitwise include: Trues and Falses, integers.

NumPy objects, by contrast, support vectorized calculations. That is, they let you perform the same operations on multiple pieces of data.

Example 3 fails because NumPy arrays (of length > 1) have no truth value as this prevents vector-based logic confusion.

Example 4 is simply a vectorized bit and operation.

Bottom Line

• If you are not dealing with arrays and are not performing math manipulations of integers, you probably want and.

• If you have vectors of truth values that you wish to combine, use numpy with &.

Answer 2

1. In Python an expression of X and Y returns Y, given that bool(X) == True or any of X or Y evaluate to False, e.g.:

   True and 20 
   >>> 20
   
   False and 20
   >>> False
   
   20 and []
   >>> []
   

2. Bitwise operator is simply not defined for lists. But it is defined for integers - operating over the binary representation of the numbers. Consider 16 (01000) and 31 (11111):

   16 & 31
   >>> 16
   

3. NumPy is not a psychic, it does not know, whether you mean that e.g. [False, False] should be equal to True in a logical expression. In this it overrides a standard Python behaviour, which is: "Any empty collection with len(collection) == 0 is False".

4. Probably an expected behaviour of NumPy's arrays's & operator.