cpython

Running Python code in parallel from Rust with rust-cpython

旧城冷巷雨未停 提交于 2021-02-08 06:14:45
问题 I'm trying to speed up a data pipeline using Rust. The pipeline contains bits of Python code that I don't want to modify, so I'm trying to run them as-is from Rust using rust-cpython and multiple threads. However, the performance is not what I expected, it's actually the same as running the python code bits sequentially in a single thread. Reading the documentation, I understand when invoking the following, you actually get a pointer to a single Python interpreter that can only be created

Running Python code in parallel from Rust with rust-cpython

我怕爱的太早我们不能终老 提交于 2021-02-08 06:13:58
问题 I'm trying to speed up a data pipeline using Rust. The pipeline contains bits of Python code that I don't want to modify, so I'm trying to run them as-is from Rust using rust-cpython and multiple threads. However, the performance is not what I expected, it's actually the same as running the python code bits sequentially in a single thread. Reading the documentation, I understand when invoking the following, you actually get a pointer to a single Python interpreter that can only be created

Recursive logging crashes Interpreter in Python 3

。_饼干妹妹 提交于 2021-02-07 17:22:19
问题 Following code logs an error and calls itself leading to stack overflow and eventually core dump in Python 3.6. >>> import logging >>> def rec(): ... logging.error("foo") ... rec() >>> rec() [1] 101641 abort (core dumped) python3 FTR, this doesn't crash Python 2.7. Attaching the error (condensed) in Python 3.6: ERROR:root:foo ... --- Logging error --- Traceback (most recent call last): ... RecursionError: maximum recursion depth exceeded in comparison ... Fatal Python error: Cannot recover

How do you pass around a void pointer between Python and C when writing an extension?

青春壹個敷衍的年華 提交于 2021-02-07 14:40:20
问题 I started on my first Python extension today and was only creating a very small wrapper around a C library as an exercise. As is typical with C libraries, you start of with an initialization function that yields a handler. You can pass that handler to functions and later you pass it to the cleanup function that frees memory. When I started writing the wrapper I basically wanted to have a way to call each native C function from python. Quickly I hit the problem that I need to return an

How do you pass around a void pointer between Python and C when writing an extension?

拥有回忆 提交于 2021-02-07 14:39:16
问题 I started on my first Python extension today and was only creating a very small wrapper around a C library as an exercise. As is typical with C libraries, you start of with an initialization function that yields a handler. You can pass that handler to functions and later you pass it to the cleanup function that frees memory. When I started writing the wrapper I basically wanted to have a way to call each native C function from python. Quickly I hit the problem that I need to return an

CPython: Dynamic module does not define module export function error

喜夏-厌秋 提交于 2021-02-05 08:13:07
问题 I just compiled my Python wrapper for C++ classes successfully. However, I am getting following messages when I am trying to load my module to the Python (through import cell ): ImportError: dynamic module does not define module export function (PyInit_cell) I checked that the system is using the Python3 on all cases so this is not a Python version problem. Below is my setup.py file: from distutils.core import setup, Extension from Cython.Build import cythonize setup(ext_modules = cythonize

Will TextIOWrapper.tell reliably return zero at the beginning of a file?

会有一股神秘感。 提交于 2021-01-29 19:13:27
问题 The docs for TextIOBase.tell, from which TextIOWrapper inherits, state Return the current stream position as an opaque number. The number does not usually represent a number of bytes in the underlying binary storage. I am wondering, whether it will always return zero in the specific case that the file pointer is at the beginning of the file? I wrote a small, non-exhaustive, script to check some obvious scenarios, which returned zero for each case: import io import os rmodes = ['a', 'a+', 'r',

Python @property.setter

亡梦爱人 提交于 2021-01-27 22:14:27
问题 The basic way of creating decorators is def my_decorator(f): def _f(*args, **kwargs): # do something using f pass return _f @my_decorator def f(...): ... But that way you cannot define decorators like @property.setter , because the name of the property (and thus the name of the decorator) is different every time. How is it @property.setter defined then? Is it possible to do something similar in Python, or is it built-in feature available only from C (implementation) level? 回答1: What you are

Tuple declaration in Python

江枫思渺然 提交于 2021-01-27 04:39:41
问题 In python, one can declare a tuple explicitly with parenthesis as such: >>> x = (0.25, 0.25, 0.25, 0.25) >>> x (0.25, 0.25, 0.25, 0.25) >>> type(x) <type 'tuple'> Alternatively, without parenthesis, python automatically packs its into a immutable tuple: >>> x = 0.25, 0.25, 0.25, 0.25 >>> x (0.25, 0.25, 0.25, 0.25) >>> type(x) <type 'tuple'> Is there a pythonic style to declare a tuple? If so, please also reference the relevant PEP or link. There's no difference in the "end-product" of

Cythonize python dictionary object

孤街浪徒 提交于 2021-01-25 07:17:35
问题 I am using cython to improve the performance of the python functions. Basically i can improve performance only in dictionary operations. So i was trying to search for any dictionary written in c, I found that cython itself contains a .pxd file which is basically wrapper for .h file at "Cython/includes/cpython". My question is if i directly use cimport dict ,how to include the above mentioned path while compiling? If i use this dict.pxd will i get any improvement in performance ? I attached