How to remove -pthread compiler flag from cython setup file

问题

In linux environment, when I run the setup script for cython, I get

gcc -pthread -B /apps/.../compiler_compat -Wl,-sysroot=/ -Wsign-compare 
-DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/ap......

for my case, I want to remove the pthread option. How do I do that thru the cython setup file? I see there are options to add compiler flags but none to remove. My setup file:

from distutils.core import setup
from Cython.Build import cythonize

from distutils.extension import Extension

extensions = [Extension("foo",sources = ["foo.pyx"])]                                 

setup(ext_modules = cythonize(extensions))

回答1:

As @DavidW has pointed out, for many options there is also an option which negates/overrides it and the last option on the command line "wins". So for example adding extra compile option -Os would overrule the default setting -O2 or -O3, because -Os would come after -O2 on the command line (-fwrapv/-fno-wrapv another example for such pairs).

However, -pthread has no such "partner" and the only chance to disable it, is to prevent its appearance on the command line altogether. The ways to do it are somewhat hacky, but is this hackiness not the reason we all use python?

distutils uses distutils.sysconfig to find the right compile/link flags. One possibility would be modify its functionality, so that -pthread is filtered out.

I choose get_config_vars, but there are also other options of course. The plan is simple:

wrap distutils.sysconfig.get_config_vars, so that -pthread is filtered out
replace distutils.sysconfig.get_config_vars with the wrapper
otherwise, setup.py is unchanged

Which leads to the following setup.py:

# manipulate get_config_vars:
# 1. step: wrap functionality and filter
from distutils.sysconfig import get_config_vars as default_get_config_vars

def remove_pthread(x):
    if type(x) is str:
        # x.replace(" -pthread ") would be probably enough...
        # but we want to make sure we make it right for every input
        if x=="-pthread":
            return ""
        if x.startswith("-pthread "):
            return remove_pthread(x[len("-pthread "):])
        if x.endswith(" -pthread"):
            return remove_pthread(x[:-len(" -pthread")])
        return x.replace(" -pthread ", " ")
    return x

def my_get_config_vars(*args):
  result = default_get_config_vars(*args)
  # sometimes result is a list and sometimes a dict:
  if type(result) is list:
     return [remove_pthread(x) for x in result]
  elif type(result) is dict:
     return {k : remove_pthread(x) for k,x in result.items()}
  else:
     raise Exception("cannot handle type"+type(result))

# 2.step: replace    
import distutils.sysconfig as dsc
dsc.get_config_vars = my_get_config_vars


# 3.step: normal setup.py

from distutils.core import setup
from Cython.Build import cythonize

from distutils.extension import Extension

extensions = [Extension("foo",sources = ["foo.pyx"])]                                 

setup(ext_modules = cythonize(extensions))

I'm not entirely sure, it is a great idea to build without -pthread and to load the resulting module in a python-interpreter which is built with -pthread - not sure it will not break in some subtle ways and work as intended.

来源：https://stackoverflow.com/questions/57046796/how-to-remove-pthread-compiler-flag-from-cython-setup-file

标签

python

cython

compiler-optimization

cythonize