Unexpected behavour when making array of 2D arrays, of similar dimension

Question

MWE:

def showArrayOfList(a,b,c):
    wlist = [np.zeros((szNext,szThis)) for (szThis,szNext) in [(a,b),(b,b),(b,b),(b,c)]]

    print "wlist:", map(np.shape,wlist)

    wArray = np.asarray(wlist)
    print "wArray:", map(np.shape,wArray)
    print "shape wArray:", shape(wArray)

np.zeros can be substituted for any other matrix function that returns a matrix given a shape

The output from the following is what I expect (and get):

In[1]: ShowArrayOfList(1,4,5)
Out[1]: wlist: [(4, 1), (4, 4), (4, 4), (5, 4)]
wArray: [(4, 1), (4, 4), (4, 4), (5, 4)]
shape wArray: (4,) #An array of 4 references(?), to arrays of various sizes

In[2]: ShowArrayOfList(5,5,5)
Out[2]: wlist: [(5, 5), (5, 5), (5, 5), (5, 5)]
wArray: [(5, 5), (5, 5), (5, 5), (5, 5)]
shape wArray: (4, 5, 5) #4 arrays of shape (5,5)

But for inputs of the form a!=b and b==c things are completely different

Int[3]: showArrayOfList(6,5,5)
Out[3]: wlist: [(5, 6), (5, 5), (5, 5), (5, 5)]
wArray: [(5,), (5,), (5,), (5,)] #Where did my second Dimension Go?
shape wArray: (4, 5)


Int[4]: showArrayOfList(2,4,4)
Out[4]:
wlist: [(4, 2), (4, 4), (4, 4), (4, 4)]
wArray: [(4,), (4,), (4,), (4,)] #Where did my second Dimension Go?
shape wArray: (4, 4)

This cause a very hard to find bug for me, With some thought, I think it has something to do with the Broadcasting system.

I would like what is going on, explained. (I have a blurry notion in my head)

---

For reference the reason I am making a array of arrays is for subtraction:
wArray=wArray-dWs is a lot clearer to read than than
wList=[w-dW, (w,dW) in zip(wList,dWs)]

Answer 1

Simply printing out the arrays should fairly quickly allow you too see what happened.

As to the question of where did the last dimension go. Since the size of that dimension has variable length. Numpy wont create a new dimension for it, it will simply create an array of objects (where the object is a list) of varying length.

In the showArrayOfList(2,4,4) case your array looks like this:

First row:
[array([ 0., 0.]) array([ 0., 0.]) array([ 0., 0.]) array([ 0., 0.])]

second to fourth row:
[array([ 0., 0., 0., 0.]) array([ 0., 0., 0., 0.]) array([ 0., 0., 0., 0.]) array([ 0., 0., 0., 0.])]

Answer 2

A more consistent way of creating wArray is to initialize it to a (4,) object array, and fill it term by term:

n = len(wlist)
wArray = np.empty((n,), dtype='O')
for i in range(n):
    wArray[i] = wlist[i]

This isn't as pretty as asarray(wlist), but it splits the 3 dimensions in the same 1,2 manner regardless of what a,b,c are.