Passing list of numpy arrays to C using cython

Question

I have a list list_of_arrays of 3D numpy arrays that I want to pass to a C function with the template

int my_func_c(double **data, int **shape,          


        

Answer 1

One alternative would be to let numpy manage your memory for you. You can do this by using numpy arrays of np.uintp which is an unsigned int with the same size as any pointer.

Unfortunately, this does require some type-casting (between "pointer sized int" and pointers) which is a good way of hiding logic errors, so I'm not 100% happy with it.

def my_func(list list_of_arrays):
    cdef int n_arrays  = len(list_of_arrays)
    cdef np.uintp_t[::1] data = np.array((n_arrays,),dtype=np.uintp)
    cdef np.uintp_t[::1] shape = np.array((n_arrays,),dtype=np.uintp)
    cdef double x;

    cdef np.ndarray[double, ndim=3, mode="c"] temp

    for i in range(n_arrays):
        temp = list_of_arrays[i]
        data[i]  = &temp[0,0,0]
        shape[i] = &(temp.shape[0])

    x = my_func_c((&data[0]), &shape[0], n_arrays)

(I should point out that I've only confirmed that it compiles and not tested it further, but the basic idea should be OK)

---

The way you've done it is probably a pretty sensible way. One slight simplification to your original code that should work

shape[i] = &(temp.shape[0])

instead of malloc and copy. I'd also recommend putting the frees in a finally block to ensure they get run.

---

Edit: @ead has helpfully pointed out that the numpy shape is stored as as np.intp_t - i.e. an signed integer big enough to fit a pointer in, which is mostly 64bit - while int is usually 32 bit. Therefore, to pass the shape without copying you'd need to change your C api. Casting help makes that mistake harder to spot ("a good way of hiding logic errors")