Initialise Cython Memoryview efficiently

北慕城南 提交于 2021-01-27 07:22:28

问题


I'm currently setting my MemoryViews in my Cython pyx file as follows:

@cython.boundscheck(False)
cdef int[:] fill_memview():
    # This happens inside a big loop so needs to be fast
    cdef int[:] x = np.empty(10)
    for i in range(10):
        x[i] = i
    return x

cdef stupid_loop():
    for i in range(10000):
        fill_memview()

When I compile the pyx file with cython -a foo.pyx the line cdef int[:] x = np.empty(10) shows in the resulting annotated html file in dark yellow (meaning it has lots of Python calls slowing things down.)

How can I instatiate my typed Memoryview better?


回答1:


See this answer for a comparison of different ways of allocating memory. If your needs are simple (just indexing) pay particular attention to 'cpython.array raw C type', you can create a cpython array for fast creation then use as_ints[i] for fast unsafe indexing, or if you really do need a memory view, the memory view on a cpython array is 3x faster than a numpy array.

Without having a bigger picture of what your code does it's difficult to offer more specific advice. For example if possible you would be better served using a two-dimensional array as it tends to be much more efficient to allocate one large chunk of memory than lots of little ones, and for instance it's much quicker to make lots of little memory view slices of one big memory view with a big hunk of allocated memory, than to create a bunch of little memory views each with their own little piece of allocated memory.




回答2:


Your memoryview is slow(er than strictly necessary) because Python needs to reference-count it. You can allocate the memory by hand using the Python/C API, but then you're responsible for freeing it when you no longer need it.

Don't do this unless you've used a profiler and are seeing an unacceptable amount of refcounting overhead. Premature optimization is never a good idea, and it's easy to introduce memory leaks or segfaults with this method.



来源:https://stackoverflow.com/questions/26843535/initialise-cython-memoryview-efficiently

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!