问题
Update 11/23/2019: This started out as a question about why I could not get f2py to work for a simple fortran wrapper. My "answer" (below) is to use ctypes instead.
Original post: I have spent the last three days trying to use f2py to interface fortran to python. I am working on windows using both cygwin and mingw. This post is about using cygwin, but I'm concerned about conflicts between the two. Here is the source multxy.f90:
subroutine multxy(x,y,z)
integer, parameter :: flt = selected_real_kind(15)
real(flt), intent(in) :: x,y
real(flt), intent(out) :: z
write(*,'(a,3g12.5)')'multxy'
write(*,'(a,3g12.5)')'multxy',x,y,x*y
z = x*y
end subroutine multxy
If I run f2py like all the examples I’ve seen:
f2py -m multxyC -c multxy.f90
It produces a link error. Amongst the tons of output, I get complaints about strtoflt128. This is a function in the gnu gcc quad math library. My flt kind is equivalent to a double in C. There is no quad math in the code, so why is it used? I tried to create an extension file to see if it would tell me anything, but it still tried to link, so no code was produced (or saved).
At this point, I built a small c program using strtofl128. It compiles and links if I include the quadmath library: gcc main.c -lquadmath
It doesn’t run correctly, but that is something I'll look into later.
I believe the actual quadmath library file is …./cwin/lib/gcc/x86_64-pc-cygwin/8.3.0/libquadmath.a. The output from running f2py shows this directory, so you would think it would link the library. I have tried running f2py with all sorts of variations of the -l option. Usually it would complain about no library found. I got one variant to work, but the link still failed.
I’ve also worked with an integer function:
integer function intfunc(n)
integer, intent(in) :: n
intfunc = n*n
end function intfunc
I have been able to create a dll and call it from python without using f2py. It seem to run through f2py just fine. It creates a file intfunc.cp37-win_amd64.pyd and a directory which contains a .dll with a huge name. I haven’t been able to call it from python yet, but I’ll keep working on that.
Questions:
- Why is f2py using quad precision? Is there an option to avoid using it?
- How do I include the correct library?
- How do I create the C extension code without invoking the linker?
- Can there be a conflict between Cygwin and MinGW? How to avoid?
- Is f2py ready for prime time? Would it be simpler to do a manual interface using the C extensions?
- Can f2py handle fortran functions? Several places have recommended using subroutines?
f2py is touted as easy to use. Perhaps it is, but only after you get it to work properly. I have been interfacing different languages for 25 years, eg. c/c++, fortran, excel/vba, matlab/octave, without nearly this much difficulty. This seems to be a program which does a lot which is hidden under the covers and that makes it difficult to troubleshoot.
One issue is that the output from running f2py is about as easy to read as a core dump. The output when I run it is:
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building extension "multxyC" sources
f2py options: []
f2py:> C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\multxyCmodule.c
creating C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7
Reading fortran codes...
Reading file 'multxy.f90' (format:free)
Post-processing...
Block: multxyC
Block: multxy
Post-processing (stage 2)...
Building modules...
Building module "multxyC"...
Constructing wrapper function "multxy"...
z = multxy(x,y)
Wrote C/API module "multxyC" to file "C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\multxyCmodule.c"
adding 'C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\fortranobject.c' to sources.
adding 'C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7' to include_dirs.
copying c:\program files\python37\lib\site-packages\numpy\f2py\src\fortranobject.c -> C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7
copying c:\program files\python37\lib\site-packages\numpy\f2py\src\fortranobject.h -> C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7
build_src: building npy-pkg config files
running build_ext
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
customize MSVCCompiler using build_ext
get_default_fcompiler: matching types: '['gnu', 'intelv', 'absoft', 'compaqv', 'intelev', 'gnu95', 'g95', 'intelvem', 'intelem', 'flang']'
customize GnuFCompiler
Could not locate executable g77
Could not locate executable f77
customize IntelVisualFCompiler
Could not locate executable ifort
Could not locate executable ifl
customize AbsoftFCompiler
Could not locate executable f90
customize CompaqVisualFCompiler
Found executable C:\cwin\bin\DF.exe
customize IntelItaniumVisualFCompiler
Could not locate executable efl
customize Gnu95FCompiler
Found executable C:\cwin\bin\gfortran.exe
Using built-in specs.
COLLECT_GCC=/usr/bin/gfortran
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-pc-cygwin/8.3.0/lto-wrapper.exe
Target: x86_64-pc-cygwin
Configured with: /cygdrive/i/szsz/tmpp/gcc/gcc-8.3.0-1.x86_64/src/gcc-8.3.0/configure --srcdir=/cygdrive/i/szsz/tmpp/gcc/gcc-8.3.0-1.x86_64/src/gcc-8.3.0 --prefix=/usr --exec-prefix=/usr --localstatedir=/var --sysconfdir=/etc --docdir=/usr/share/doc/gcc --htmldir=/usr/share/doc/gcc/html -C --build=x86_64-pc-cygwin --host=x86_64-pc-cygwin --target=x86_64-pc-cygwin --without-libiconv-prefix --without-libintl-prefix --libexecdir=/usr/lib --enable-shared --enable-shared-libgcc --enable-static --enable-version-specific-runtime-libs --enable-bootstrap --enable-__cxa_atexit --with-dwarf2 --with-tune=generic --enable-languages=ada,c,c++,fortran,lto,objc,obj-c++ --enable-graphite --enable-threads=posix --enable-libatomic --enable-libgomp --enable-libitm --enable-libquadmath --enable-libquadmath-support --disable-libssp --enable-libada --disable-symvers --with-gnu-ld --with-gnu-as --with-cloog-include=/usr/include/cloog-isl --without-libiconv-prefix --without-libintl-prefix --with-system-zlib --enable-linker-build-id --with-default-libstdcxx-abi=gcc4-compatible --enable-libstdcxx-filesystem-ts
Thread model: posix
gcc version 8.3.0 (GCC)
customize Gnu95FCompiler
Using built-in specs.
COLLECT_GCC=/usr/bin/gfortran
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-pc-cygwin/8.3.0/lto-wrapper.exe
Target: x86_64-pc-cygwin
Configured with: /cygdrive/i/szsz/tmpp/gcc/gcc-8.3.0-1.x86_64/src/gcc-8.3.0/configure --srcdir=/cygdrive/i/szsz/tmpp/gcc/gcc-8.3.0-1.x86_64/src/gcc-8.3.0 --prefix=/usr --exec-prefix=/usr --localstatedir=/var --sysconfdir=/etc --docdir=/usr/share/doc/gcc --htmldir=/usr/share/doc/gcc/html -C --build=x86_64-pc-cygwin --host=x86_64-pc-cygwin --target=x86_64-pc-cygwin --without-libiconv-prefix --without-libintl-prefix --libexecdir=/usr/lib --enable-shared --enable-shared-libgcc --enable-static --enable-version-specific-runtime-libs --enable-bootstrap --enable-__cxa_atexit --with-dwarf2 --with-tune=generic --enable-languages=ada,c,c++,fortran,lto,objc,obj-c++ --enable-graphite --enable-threads=posix --enable-libatomic --enable-libgomp --enable-libitm --enable-libquadmath --enable-libquadmath-support --disable-libssp --enable-libada --disable-symvers --with-gnu-ld --with-gnu-as --with-cloog-include=/usr/include/cloog-isl --without-libiconv-prefix --without-libintl-prefix --with-system-zlib --enable-linker-build-id --with-default-libstdcxx-abi=gcc4-compatible --enable-libstdcxx-filesystem-ts
Thread model: posix
gcc version 8.3.0 (GCC)
customize Gnu95FCompiler using build_ext
building 'multxyC' extension
compiling C sources
creating C:\cwin\tmp\tmprix0z7i7\Release\cwin
creating C:\cwin\tmp\tmprix0z7i7\Release\cwin\tmp
creating C:\cwin\tmp\tmprix0z7i7\Release\cwin\tmp\tmprix0z7i7
creating C:\cwin\tmp\tmprix0z7i7\Release\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MT -IC:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7 -Ic:\program files\python37\lib\site-packages\numpy\core\include -Ic:\program files\python37\include -Ic:\program files\python37\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\ATLMFC\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\include -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt /TcC:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\multxyCmodule.c /FoC:\cwin\tmp\tmprix0z7i7\Release\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\multxyCmodule.obj
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MT -IC:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7 -Ic:\program files\python37\lib\site-packages\numpy\core\include -Ic:\program files\python37\include -Ic:\program files\python37\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\ATLMFC\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\include -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt /TcC:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\fortranobject.c /FoC:\cwin\tmp\tmprix0z7i7\Release\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\fortranobject.obj
compiling Fortran sources
Fortran f77 compiler: C:\cwin\bin\gfortran.exe -Wall -g -ffixed-form -fno-second-underscore -O3 -funroll-loops
Fortran f90 compiler: C:\cwin\bin\gfortran.exe -Wall -g -fno-second-underscore -O3 -funroll-loops
Fortran fix compiler: C:\cwin\bin\gfortran.exe -Wall -g -ffixed-form -fno-second-underscore -Wall -g -fno-second-underscore -O3 -funroll-loops
compile options: '-IC:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7 -Ic:\program files\python37\lib\site-packages\numpy\core\include -Ic:\program files\python37\include -Ic:\program files\python37\include -c'
gfortran.exe:f90: multxy.f90
C:\cwin\bin\gfortran.exe -Wall -g -Wall -g -shared ..\..\..\..\cwin\tmp\tmprix0z7i7\Release\multxy.o -L/usr/lib/gcc/x86_64-pc-cygwin/8.3.0 -Lc:\program files\python37\libs -Lc:\program files\python37\PCbuild\amd64 -o C:\cwin\tmp\tmprix0z7i7\Release\.libs\libmultxy.J7YCD6VUIR3DWPQ3PSLGVWZTCQ2KMJO6.gfortran-win_amd64.dll -Wl,--allow-multiple-definition -Wl,--output-def,C:\cwin\tmp\tmprix0z7i7\Release\libmultxy.J7YCD6VUIR3DWPQ3PSLGVWZTCQ2KMJO6.gfortran-win_amd64.def -Wl,--export-all-symbols -Wl,--enable-auto-import -static -mlong-double-64
/usr/lib/gcc/x86_64-pc-cygwin/8.3.0/../../../../x86_64-pc-cygwin/bin/ld: /usr/lib/gcc/x86_64-pc-cygwin/8.3.0/libgfortran.a(read.o): in function `_gfortrani_convert_real':
/usr/src/debug/gcc-8.3.0-1/libgfortran/io/read.c:173:(.text$_gfortrani_convert_real+0x85): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `strtoflt128'
/usr/lib/gcc/x86_64-pc-cygwin/8.3.0/../../../../x86_64-pc-cygwin/bin/ld: /usr/lib/gcc/x86_64-pc-cygwin/8.3.0/libgfortran.a(read.o): in function `_gfortrani_convert_infnan':
/usr/src/debug/gcc-8.3.0-1/libgfortran/io/read.c:249:(.text$_gfortrani_convert_infnan+0x5c): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `strtoflt128'
collect2: error: ld returned 1 exit status
error: Command "C:\cwin\bin\gfortran.exe -Wall -g -Wall -g -shared ..\..\..\..\cwin\tmp\tmprix0z7i7\Release\multxy.o -L/usr/lib/gcc/x86_64-pc-cygwin/8.3.0 -Lc:\program files\python37\libs -Lc:\program files\python37\PCbuild\amd64 -o C:\cwin\tmp\tmprix0z7i7\Release\.libs\libmultxy.J7YCD6VUIR3DWPQ3PSLGVWZTCQ2KMJO6.gfortran-win_amd64.dll -Wl,--allow-multiple-definition -Wl,--output-def,C:\cwin\tmp\tmprix0z7i7\Release\libmultxy.J7YCD6VUIR3DWPQ3PSLGVWZTCQ2KMJO6.gfortran-win_amd64.def -Wl,--export-all-symbols -Wl,--enable-auto-import -static -mlong-double-64" failed with exit status 1
回答1:
I did not exactly figure out the problem with f2py, so please don't ding me for this nonanswer. All I wanted was to put a simple wrapper around some fortran. I solved that problem though, so this post is my answer. I am posting it in hopes it may help someone else with a similar problem.
After the post, I decided I needed to drop down to a lower level to figure out what was going on. Since f2py is built on the C-api, I decided to work with it directly. C - fortran interoperability is well defined. I got some simple code to work, but I was scared off when I started looking at passing arrays and read about reference counting. So, I decided the C-api it was too low level for my needs. Cython appears oriented toward writing Python-like code that can be compiled. I finally settled on ctypes, since all I really need is to convert the internal python types and then call (what it thinks is) c code. There are examples around but none seemed to fit exactly, so I will post some test code in hopes it may help others. The example shows passing ints, doubles and arrays back and forth by both functions and subroutines. The same code works in windows if .so is replaced with .dll.This information was helpful - How to dereference a memory location from python ctypes?
The fortran is:
Integer Function intfunc(n)
integer, intent(in) :: n
intfunc = n*n
end Function intfunc
! ---------------------------------------------------------------------------------------
subroutine Asub(acf,af)
include 'defs.fi' ! integer, parameter :: float = selected_real_kind(12)
Real(float), intent(in) :: acf
Real(float), intent(out) :: af
if(acf < 0.49)then
af = 0.37464 + acf*(1.54226 - 0.26992*acf)
else
af = 0.379642 + acf*(1.48503 + acf*(-0.164423 + 0.016666*acf))
endif
end subroutine Asub
! ---------------------------------------------------------------------------------------
Function Afactor(acf) result(af)
include 'defs.fi'
Real(float), intent(in) :: acf
Real(float) :: af
if(acf < 0.49)then
af = 0.37464 + acf*(1.54226 - 0.26992*acf)
else
af = 0.379642 + acf*(1.48503 + acf*(-0.164423 + 0.016666*acf))
endif
End Function Afactor
! ---------------------------------------------------------------------------------------
Function MultXY(x,y) result(z)
include 'defs.fi'
Real(float), intent(in) :: x,y
Real(float)::z
z = x*y
End Function MultXY
! ---------------------------------------------------------------------------------------
subroutine arrays(n,m,a)
include 'defs.fi'
integer, intent(in) :: n,m
real(float), intent(out) :: a(n,m)
integer :: i,j
do j=1,m
do i=1,n
a(i,j) = real(100*i+j,float)
enddo
enddo
end subroutine arrays
The code is compiled into a shared library by:
gfortran -shared -o code.so code.f90 -fPIC -Xlinker -Map=code.map
It can be run with the following python code. A number of the steps in this code could be combined, but I thought it was more illustrative to break them down. I don't completely understand the function prototypes (argtypes & restypes). They seem to allow some shortcuts, such that Python performs the appropriate cast. Can someone elaborate?
import ctypes as ct
import numpy as np
from numpy.ctypeslib import ndpointer
flib = ct.CDLL('code.so')
flib.asub_.argtypes = [ct.POINTER(ct.c_double),ct.POINTER(ct.c_double)]
flib.afactor_.argtypes = [ct.POINTER(ct.c_double)]
flib.afactor_.restype = ct.c_double
flib.multxy_.argtypes = [ct.POINTER(ct.c_double),ct.POINTER(ct.c_double)]
flib.multxy_.restype = ct.c_double
flib.arrays_.argtypes = [ct.POINTER(ct.c_int),ct.POINTER(ct.c_int),ndpointer(ct.c_double)]
def main():
nval = 4
nptr = ct.pointer(ct.c_int(nval))
# integer function intfunc(n)
nx = flib.intfunc_(nptr)
print('intfunc(',nval,') =',nx)
x = 0.35
xc = ct.c_double(x)
yc = ct.c_double(0.50)
zc = ct.c_double(0.0)
# subroutine Asub(acf,af)
flib.asub_(ct.byref(xc),ct.byref(zc))
z = zc.value
x = xc.value
y = yc.value
print(' x,z =',x,z)
# Function Afactor(acf) result(af)
z = flib.afactor_(ct.byref(xc))
print(' x,z =',x,z)
# Function MultXY(x,y) result(z)
z = flib.multxy_(ct.byref(xc),ct.byref(yc))
print('multxy(',x,y,') =',z)
n = 3
m = 4
nc = ct.c_int(n)
mc = ct.c_int(m)
a = np.empty((m,n),dtype=np.double) # m,n not n,m
# subroutine arrays(n,m,a)
flib.arrays_(ct.byref(nc),ct.byref(mc),a)
for i in range(n): # note reversal of indices
for j in range(m):
print('i,j,a(j,i) =',i+1,j+1,a[j,i])
main()
The output is:
intfunc( 4 ) = 16
x,z = 0.35 0.8813658067584037
x,z = 0.35 0.8813658067584037
multxy( 0.35 0.5 ) = 0.175
i,j,a(j,i) = 1 1 101.0
i,j,a(j,i) = 1 2 102.0
i,j,a(j,i) = 1 3 103.0
i,j,a(j,i) = 1 4 104.0
i,j,a(j,i) = 2 1 201.0
i,j,a(j,i) = 2 2 202.0
i,j,a(j,i) = 2 3 203.0
i,j,a(j,i) = 2 4 204.0
i,j,a(j,i) = 3 1 301.0
i,j,a(j,i) = 3 2 302.0
i,j,a(j,i) = 3 3 303.0
i,j,a(j,i) = 3 4 304.0
来源：https://stackoverflow.com/questions/58923637/f2py-linking-quadmath-libraries-use-ctypes-for-fortran-wrapper-instead