How do I make a grid of empty lists in numpy that 'accepts' appending?

Question

I am trying to use numpy.append but something goes wrong and it just doesn\'t make sence to me anymore. Can someone explain why I am getting an error?

>>&g         


        

Answer 1

First of all, stop updating your question when someone answers it correctly. Accept/upvote an answer and ask a new question when you are ready. SO is a Q&A site that is meant to help future visitors viewing your question, and not just you. Invalidating all the good answers by changing the context completely for your own benefit defeats the purpose of this site, to say the least.

Secondly, np.array([[], [3], []]) ends up with dtype=object because it is a ragged array. [[], [], []] and [[3], [3], [3]], having a uniform length in all elements across all dimensions, would produce numerical arrays.

np.zeros((6,),dtype=object) produces an empty array of object references (containing NULLs). When you fill it with [], you are filling it with a reference to the same python list in every element. numpy has no knowledge of what object you are passing to ndarray.fill, so it does not call the constructor on the list type for each element as you seem to be expecting. It just copies the reference that you passed in six times. After that, it should be clear why changing the contents of that one list makes it appear that all the array elements have changed.

Answer 2

I know this is a rather unusual use case but in fact it can be very handy to have an ndarray holding lists in each cell. Imho numpy should allow to pass a lambda function to the fill mehtod. But to achive what you want this is what I do:

m = np.empty((12, 12), dtype=object)
for i in np.ndindex(m.shape): m[i] = []

Answer 3

Don't just look at the shape; check the dtype, and if object, the nature of the elements

In [282]: np.array([[], [], []])
Out[282]: array([], shape=(3, 0), dtype=float64)

A 2d array of floats. np.array tries to make a multidimensional array of numbers; it's only when it can't do that it makes an object array.

In [283]: b=np.array([[],[3],[]])
In [284]: b
Out[284]: array([[], [3], []], dtype=object)

Here the 3 sublists have different size, so it can't make a 2d array; the result is an object array, where the objects are lists, and have the append method.

In [286]: c=np.array((3,0), object)
In [287]: c
Out[287]: array([3, 0], dtype=object)

This is a (2,) object array; the 2 elements are numbers. Numbers don't have an append method.

In [288]: np.empty((3,1))
Out[288]: 
array([[ 0.],
       [ 0.],
       [ 0.]])

A (3,1) array of floats. No append method for numbers or arrays.

In [289]: np.empty((3,0))
Out[289]: array([], shape=(3, 0), dtype=float64)

Another 2d array of floats

In [290]: np.empty((6,1),object)
Out[290]: 
array([[None],
       [None],
       [None],
       [None],
       [None],
       [None]], dtype=object)

2d array of dtype object. In this case they are initialized to None. Again no append method.

More on making an array of lists

dimensions of array of arrays in numpy

and

How to keep numpy from broadcasting when creating an object array of different shaped arrays

---

In [305]: d=np.empty((3,),object)
In [306]: d
Out[306]: array([None, None, None], dtype=object)
In [307]: d.fill([])
In [308]: d
Out[308]: array([[], [], []], dtype=object)   # array of lists
In [309]: d[0].append([1,2,3])
In [310]: d
Out[310]: array([[[1, 2, 3]], [[1, 2, 3]], [[1, 2, 3]]], dtype=object)

But oops - those lists are all the same object (pointer) :( I have to put a different list in each element. Now I can append to them individually.

In [311]: d[...]=[[],[1,2,3],[2]]
In [312]: d
Out[312]: array([[], [1, 2, 3], [2]], dtype=object)
In [313]: d[0].append([2,3])
In [314]: d
Out[314]: array([[[2, 3]], [1, 2, 3], [2]], dtype=object)

I think you have to bite the bullet and use a list to initialize an object array of lists. There isn't a short cut:

In [319]: d=np.empty((3,),object)
In [320]: d[...]=[[] for _ in range(3)]
In [321]: d
Out[321]: array([[], [], []], dtype=object)
In [323]: d
Out[323]: array([[], [3], []], dtype=object)