问题
I am trying to create a simple Yocto Python Recipe for Python 3 PyTorch. The target is an SD Card Image for the NVIDIA Jetson Nano produced by Yocto from the meta-tegra layer. I can successfully compile and boot an image from meta-tegra without these recipes.
NVIDIA themselves have compiled and released the ".whl" Python packages and they are found here: https://devtalk.nvidia.com/default/topic/1048776/official-tensorflow-for-jetson-nano-/
Using their build instructions, I tried to write a Bitbake recipe to install PyTorch as follows:
SUMMARY = "Facebook PyTorch AI"
DESCRIPTION = "Facebook PyTorch AI"
HOMEPAGE = "https://pytorch.org/"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE;md5=acf4d595f99e159bf31797aa872aef57"
inherit pypi
inherit setuptools3
#BBCLASSEXTEND = "native nativesdk"
DEPENDS += "python3-pytest-runner-native python3-pyyaml-native cmake-native"
do_configure_prepend() {
USE_NCCL=0
USE_DISTRIBUTED=0
TORCH_CUDA_ARCH_LIST="5.3;6.2;7.2"
}
do_compile_prepend() {
USE_NCCL=0
USE_DISTRIBUTED=0
TORCH_CUDA_ARCH_LIST="5.3;6.2;7.2"
}
SRC_URI = "gitsm://github.com/pytorch/pytorch.git;protocol=https"
SRCREV = "a3346e100e7f4e7ec90f18b7befcccc47d5a1c82"
S = "${WORKDIR}/git"
The problem is that I hit the following error. I think it has to do with my environment variables not being picked up becuase the error includes "-DUSE_CUDA=False", however when I look at the bitbake environment ( bitbake -e ), it seems to be there.
Here is the error:
**NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
ERROR: python3-torch-1.1.0-r0 do_compile: 'python3 setup.py build ' execution failed.
ERROR: python3-torch-1.1.0-r0 do_compile: Function failed: do_compile (log file is located at /home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/temp/log.do_compile.27625)
ERROR: Logfile of failure stored in: /home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/temp/log.do_compile.27625
Log data follows:
| DEBUG: Executing shell function do_compile
| -- The CXX compiler identification is GNU 7.2.1
| -- The C compiler identification is GNU 7.2.1
| -- Check for working CXX compiler: /home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux/aarch64-poky-linux-g++
| -- Check for working CXX compiler: /home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux/aarch64-poky-linux-g++ -- works
| -- Detecting CXX compiler ABI info
| -- Detecting CXX compiler ABI info - done
| -- Detecting CXX compile features
| -- Detecting CXX compile features - done
| -- Check for working C compiler: /home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux/aarch64-poky-linux-gcc
| -- Check for working C compiler: /home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux/aarch64-poky-linux-gcc -- works
| -- Detecting C compiler ABI info
| -- Detecting C compiler ABI info - done
| -- Detecting C compile features
| -- Detecting C compile features - done
| -- Not forcing any particular BLAS to be found
| -- Performing Test COMPILER_WORKS
| -- Performing Test COMPILER_WORKS - Failed
| CMake Error at cmake/MiscCheck.cmake:52 (message):
| Could not run a simple program built with your compiler. If you are trying
| to use -fsanitize=address, make sure libasan is properly installed on your
| system (you can confirm if the problem is this by attempting to build and
| run a small program.)
| Call Stack (most recent call first):
| CMakeLists.txt:271 (include)
|
|
| -- Configuring incomplete, errors occurred!
| See also "/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/git/build/CMakeFiles/CMakeOutput.log".
| See also "/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/git/build/CMakeFiles/CMakeError.log".
| Building wheel torch-1.2.0a0+a3346e1
| -- Building version 1.2.0a0+a3346e1
| cmake -DBUILD_AR=ar -DBUILD_AS=as -DBUILD_CC=gcc -DBUILD_CCLD=gcc -DBUILD_CFLAGS=-isystem/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/include -O2 -pipe -DBUILD_CPP=gcc -E -DBUILD_CPPFLAGS=-isystem/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/include -DBUILD_CXX=g++ -DBUILD_CXXFLAGS=-isystem/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/include -O2 -pipe -DBUILD_FC=gfortran -DBUILD_LD=ld -DBUILD_LDFLAGS=-L/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/lib -L/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/lib -Wl,--enable-new-dtags -Wl,-rpath-link,/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/lib -Wl,-rpath-link,/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/lib -Wl,-rpath,/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/lib -Wl,-rpath,/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/lib -Wl,-O1 -Wl,--allow-shlib-undefined -Wl,--dynamic-linker=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2 -DBUILD_NM=nm -DBUILD_PYTHON=True -DBUILD_RANLIB=ranlib -DBUILD_STRIP=strip -DBUILD_TEST=True -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS= -O2 -pipe -g -feliminate-unused-debug-types -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0=/usr/src/debug/python3-torch/1.1.0-r0 -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot= -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native= -DCMAKE_C_FLAGS= -O2 -pipe -g -feliminate-unused-debug-types -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0=/usr/src/debug/python3-torch/1.1.0-r0 -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot= -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native= -DCMAKE_EXE_LINKER_FLAGS=-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -fstack-protector-strong -Wl,-z,relro,-z,now -DCMAKE_INSTALL_PREFIX=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/git/torch -DCMAKE_PREFIX_PATH=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot/usr/lib/python3.7/site-packages -DCMAKE_SHARED_LINKER_FLAGS=-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -fstack-protector-strong -Wl,-z,relro,-z,now -DINSTALL_TEST=True -DPYTHON_EXECUTABLE=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/bin/python3-native/python3 -DPYTHON_INCLUDE_DIR=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot/usr/include/python3.7m -DPYTHON_LIBRARY=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot/usr/lib/libpython3.7m.so.1.0 -DTORCH_BUILD_VERSION=1.2.0a0+a3346e1 -DUSE_CUDA=False -DUSE_DISTRIBUTED=True -DUSE_FBGEMM=True -DUSE_NCCL=False -DUSE_NUMPY=True -DUSE_QNNPACK=True -DUSE_SYSTEM_EIGEN_INSTALL=OFF -DUSE_SYSTEM_NCCL=False /home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/git
| Traceback (most recent call last):
| File "setup.py", line 749, in <module>
| build_deps()
| File "setup.py", line 323, in build_deps
| cmake=cmake)
| File "/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/git/tools/build_pytorch_libs.py", line 61, in build_caffe2
| rerun_cmake)
| File "/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/git/tools/setup_helpers/cmake.py", line 318, in generate
| self.run(args, env=my_env)
| File "/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/git/tools/setup_helpers/cmake.py", line 93, in run
| check_call(command, cwd=self.build_dir, env=env)
| File "/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/lib/python3.7/subprocess.py", line 347, in check_call
| raise CalledProcessError(retcode, cmd)
| subprocess.CalledProcessError: Command '['cmake', '-DBUILD_AR=ar', '-DBUILD_AS=as ', '-DBUILD_CC=gcc ', '-DBUILD_CCLD=gcc ', '-DBUILD_CFLAGS=-isystem/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/include -O2 -pipe', '-DBUILD_CPP=gcc -E', '-DBUILD_CPPFLAGS=-isystem/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/include', '-DBUILD_CXX=g++ ', '-DBUILD_CXXFLAGS=-isystem/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/include -O2 -pipe', '-DBUILD_FC=gfortran ', '-DBUILD_LD=ld ', '-DBUILD_LDFLAGS=-L/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/lib -L/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/lib -Wl,--enable-new-dtags -Wl,-rpath-link,/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/lib -Wl,-rpath-link,/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/lib -Wl,-rpath,/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/lib -Wl,-rpath,/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/lib -Wl,-O1 -Wl,--allow-shlib-undefined -Wl,--dynamic-linker=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2', '-DBUILD_NM=nm', '-DBUILD_PYTHON=True', '-DBUILD_RANLIB=ranlib', '-DBUILD_STRIP=strip', '-DBUILD_TEST=True', '-DCMAKE_BUILD_TYPE=Release', '-DCMAKE_CXX_FLAGS= -O2 -pipe -g -feliminate-unused-debug-types -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0=/usr/src/debug/python3-torch/1.1.0-r0 -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot= -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native= ', '-DCMAKE_C_FLAGS= -O2 -pipe -g -feliminate-unused-debug-types -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0=/usr/src/debug/python3-torch/1.1.0-r0 -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot= -fdebug-prefix-map=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native= ', '-DCMAKE_EXE_LINKER_FLAGS=-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -fstack-protector-strong -Wl,-z,relro,-z,now', '-DCMAKE_INSTALL_PREFIX=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/git/torch', '-DCMAKE_PREFIX_PATH=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot/usr/lib/python3.7/site-packages', '-DCMAKE_SHARED_LINKER_FLAGS=-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -fstack-protector-strong -Wl,-z,relro,-z,now', '-DINSTALL_TEST=True', '-DPYTHON_EXECUTABLE=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot-native/usr/bin/python3-native/python3', '-DPYTHON_INCLUDE_DIR=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot/usr/include/python3.7m', '-DPYTHON_LIBRARY=/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/recipe-sysroot/usr/lib/libpython3.7m.so.1.0', '-DTORCH_BUILD_VERSION=1.2.0a0+a3346e1', '-DUSE_CUDA=False', '-DUSE_DISTRIBUTED=True', '-DUSE_FBGEMM=True', '-DUSE_NCCL=False', '-DUSE_NUMPY=True', '-DUSE_QNNPACK=True', '-DUSE_SYSTEM_EIGEN_INSTALL=OFF', '-DUSE_SYSTEM_NCCL=False', '/home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/git']' returned non-zero exit status 1.
| ERROR: 'python3 setup.py build ' execution failed.
| WARNING: exit code 1 from a shell command.
| ERROR: Function failed: do_compile (log file is located at /home/ubuntu/Desktop/comp-jetson-yocto/build/tmp/work/aarch64-poky-linux/python3-torch/1.1.0-r0/temp/log.do_compile.27625)
ERROR: Task (/home/ubuntu/Desktop/comp-jetson-yocto/layers/meta-comp/recipes-python/python3-torch/python3-torch_1.1.0.bb:do_compile) failed with exit code '1'
NOTE: Tasks Summary: Attempted 1770 tasks of which 1763 didn't need to be rerun and 1 failed.
Summary: 1 task failed:
/home/ubuntu/Desktop/comp-jetson-yocto/layers/meta-comp/recipes-python/python3-torch/python3-torch_1.1.0.bb:do_compile
Summary: There was 1 WARNING message shown.
Summary: There were 2 ERROR messages shown, returning a non-zero exit code.**
来源：https://stackoverflow.com/questions/56956555/yocto-warrior-bitbake-recipe-for-pytorch-for-nvidia-jetson-nano