c malloc array pointer return in cython

Question

How does one to return a malloc array pointer (or numpy array pointer) in cython back to python3, efficiently.

The cython code works perfectly as long as I don\'t re

Answer 1

For gcc 5+ using the C-11 standard ( gcc -std=gnu11 . . . ), the syntax for multi-dimensional malloc and calloc arrays has changed significantly.

A main() procedure, for creating a 2-D, double, complex calloc array r[n][n] for n = 1024 is now:

long n = 1024;
complex double (*r)[n] = calloc(n, sizeof *r);

An example of a Gaussian random number generator randn_box_muller() using a pointer to this calloc array r[n][n] is:

inline static void randn_box_muller(long n, complex double r[][n])
{
    long i, j; 
    register double x, y;

    for(i = 0; i < n; i++){
        for(j = 0; j < n; j++){  
            x = 2.*M_PI*dsfmt_genrand_close_open(&dsfmt);
            y = sqrt(-2.*log(dsfmt_genrand_close_open(&dsfmt)));
            r[i][j] = (cos(x) + I*sin(x))*y;
        }
     }
     return;
}

This relatively new calloc allocation syntax is a bit strange. It works well for 1, 2 and even n dimensional calloc and malloc arrays. Hopefully this will also work in conjunction with Python3. I hope to be testing this shortly.

Answer 2

Numpy C API

Your question is similar to this post.

You can use the function below to pass a C pointer to Numpy array. The memory will be freed automatically when the Numpy array is recycled. If you want free the pointer mamully, you should not set NPY_OWNDATA flag.

import numpy as np
cimport numpy as np

cdef pointer_to_numpy_array_complex128(void * ptr, np.npy_intp size):
    '''Convert c pointer to numpy array.
    The memory will be freed as soon as the ndarray is deallocated.
    '''
    cdef extern from "numpy/arrayobject.h":
        void PyArray_ENABLEFLAGS(np.ndarray arr, int flags)
    cdef np.ndarray[np.complex128, ndim=1] arr = \
            np.PyArray_SimpleNewFromData(1, &size, np.NPY_COMPLEX128, ptr)
    PyArray_ENABLEFLAGS(arr, np.NPY_OWNDATA)
    return arr

For reference:

• PyArray_SimpleNewFromData
• Numpy Data Type API

Cython Typed Memoryviews

Of couse, you can also use cython memoryview.

import numpy as np
cimport numpy as np

cdef np.complex128_t[:,:] view = <np.complex128_t[:n,:n]> c_pointer
numpy_arr = np.asarray(view)

The code above will transfer C pointer to a numpy array. However this would not free memory automaticlly, you have to free the memory by yourself or it would lead to memory leak!

Answer 3

I think the best approach is to pass the pointer of an existing array created in Python via NumPy to Cython, otherwise it seems you have to copy the content of the array created by malloc to another array, like demonstrated in this toy example:

import numpy as np
cimport numpy as np

from libc.stdlib cimport malloc, free

def main():
  cdef int i, n=40
  cdef double complex *r
  cdef np.ndarray[np.complex128_t, ndim=1] a
  a = np.zeros(n*n, dtype=np.complex128)
  r = <double complex *>malloc(n*n*sizeof(double complex))
  for i in range(n*n):
      r[i] = 1.
  for i in range(n*n):
      a[i] = r[i]
  free(r)
  return a