How to populate an existing numpy array with specific dtype

Question

Let\'s say that I have this initial numpy array with some fixed dtype:

array = numpy.array([(1, \'a\'), (2, \'b\')],
                    numpy.dtype([(\'idfield\         


        

Answer 1

Like @juanpa.arrivillaga commented, it's cleaner to define your dtype only once:

array_dt = np.dtype([
    ('idfield', np.int32),
    ('textfield', '|S256')
])

Then define your second list of values as an array and then concatenate

array2 = np.array(value, array_dt)                                     
array = np.concatenate([array, array2])

Answer 2

np.append is a simple cover to np.concatenate

def append(arr, values, axis=None):
    arr = asanyarray(arr)
    if axis is None:
        if arr.ndim != 1:
            arr = arr.ravel()
        values = ravel(values)
        axis = arr.ndim-1
    return concatenate((arr, values), axis=axis)

In [89]: dt = np.dtype('U5,int')
In [90]: arr = np.array([('one',1)], dtype=dt)
In [91]: np.append(arr, ('two',2))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-91-bc17d9ad4a77> in <module>()
----> 1 np.append(arr, ('two',2))
 ...
-> 5166     return concatenate((arr, values), axis=axis)

TypeError: invalid type promotion

In this case it does

In [92]: np.ravel(('two',2))
Out[92]: array(['two', '2'], dtype='<U3')

turning the tuple into a 2 element string dtype array. Now concatenate tries to join an array of dt with the U3 array, and it can't. There's nothing in the append uses the arr.dtype as the basis for turning values into an array. You need to do that yourself. numpy can only do so much to infer you intentions. :)

So if you specify common the dtype it works:

In [93]: np.append(arr, np.array(('two',2),dt))
Out[93]: array([('one', 1), ('two', 2)], dtype=[('f0', '<U5'), ('f1', '<i4')])

I dislike append because new users often misuse it. Usually they think of it as a list append clone, which it is not.

But it does have one advantage - it promotes the dimensions of 0d inputs:

In [94]: np.concatenate([arr, np.array(('two',2),dt)])
...
ValueError: all the input arrays must have same number of dimensions

Making the 2nd array 1d works:

In [95]: np.concatenate([arr, np.array([('two',2)],dt)])
Out[95]: array([('one', 1), ('two', 2)], dtype=[('f0', '<U5'), ('f1', '<i4')])

append hides the dimensional adjustment that concatenate needs.

But where possible it is better to create a list of arrays (or tuples) and do concatenate just once:

In [96]: alist = [('one',1),('two',2),('three',3)]
In [97]: ll = [np.array([x],dt) for x in alist]
In [98]: ll
Out[98]: 
[array([('one', 1)], dtype=[('f0', '<U5'), ('f1', '<i4')]),
 array([('two', 2)], dtype=[('f0', '<U5'), ('f1', '<i4')]),
 array([('three', 3)], dtype=[('f0', '<U5'), ('f1', '<i4')])]

In [100]: np.concatenate(ll)
Out[100]: 
array([('one', 1), ('two', 2), ('three', 3)],
      dtype=[('f0', '<U5'), ('f1', '<i4')])

But making the array directly from a list of tuples is even better:

In [101]: np.array(alist, dt)
Out[101]: 
array([('one', 1), ('two', 2), ('three', 3)],
      dtype=[('f0', '<U5'), ('f1', '<i4')])