How to install pyarrow on an Alpine Docker image?

问题

I am trying to install pyarrow using pip in my alpine docker image, but pip is unable to find the package.

I'm using the following Dockerfile:

FROM python:3.6-alpine3.7

RUN apk add --no-cache musl-dev linux-headers g++

RUN pip install pyarrow

output:

Sending build context to Docker daemon  4.096kB
Step 1/3 : FROM python:3.6-alpine3.7
3.6-alpine3.7: Pulling from library/python
ff3a5c916c92: Pull complete
471170bb1257: Pull complete
d487cc70216e: Pull complete
9358b3ca3321: Pull complete
78b9945f52f1: Pull complete
Digest: 
sha256:10bd7a59cfac2a784bedd1e6d89887995559f00b61f005a101845ed736bed779
Status: Downloaded newer image for python:3.6-alpine3.7
---> 4b00a94b6f26
Step 2/3 : RUN apk add --no-cache musl-dev linux-headers g++
---> Running in d024d0b961a6
fetch http://dl-
cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-
cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
(1/18) Upgrading musl (1.1.18-r2 -> 1.1.18-r3)
(2/18) Installing libgcc (6.4.0-r5)
(3/18) Installing libstdc++ (6.4.0-r5)
(4/18) Installing binutils-libs (2.28-r3)
(5/18) Installing binutils (2.28-r3)
(6/18) Installing gmp (6.1.2-r1)
(7/18) Installing isl (0.18-r0)
(8/18) Installing libgomp (6.4.0-r5)
(9/18) Installing libatomic (6.4.0-r5)
(10/18) Installing pkgconf (1.3.10-r0)
(11/18) Installing mpfr3 (3.1.5-r1)
(12/18) Installing mpc1 (1.0.3-r1)
(13/18) Installing gcc (6.4.0-r5)
(14/18) Installing musl-dev (1.1.18-r3)
(15/18) Installing libc-dev (0.7.1-r0)
(16/18) Installing g++ (6.4.0-r5)
(17/18) Upgrading musl-utils (1.1.18-r2 -> 1.1.18-r3)
(18/18) Installing linux-headers (4.4.6-r2)
Executing busybox-1.27.2-r7.trigger
OK: 190 MiB in 51 packages
Removing intermediate container d024d0b961a6
---> 8039ae62bbe7
Step 3/3 : RUN pip install pyarrow
---> Running in ecd1d7bc630c
Collecting pyarrow
 Could not find a version that satisfies the requirement pyarrow (from 
 versions: )
No matching distribution found for pyarrow
The command '/bin/sh -c pip install pyarrow' returned a non-zero code: 1

Has anyone community able to install pyarrow in alpine container?

回答1:

No, not that I am aware. At the moment we are only providing glibc-based Python wheels for Linux users. To use pyarrow on Alpine Linux you would need to build from source -- I am not aware of anyone having tested the library on this platform, though.

回答2:

As Wes said above you have to build from source but there were some issues in even building it from source, now we have a solution for it.

FROM python:3.7-alpine3.8

RUN apk add --no-cache \
            git \
            build-base \
            cmake \
            bash \
            jemalloc-dev \
            boost-dev \
            autoconf \
            zlib-dev \
            flex \
            bison

RUN pip install six numpy pandas cython pytest

RUN git clone https://github.com/apache/arrow.git

RUN mkdir /arrow/cpp/build    
WORKDIR /arrow/cpp/build

ENV ARROW_BUILD_TYPE=release
ENV ARROW_HOME=/usr/local
ENV PARQUET_HOME=/usr/local

#disable backtrace
RUN sed -i -e '/_EXECINFO_H/,/endif/d' -e '/execinfo/d' ../src/arrow/util/logging.cc

RUN cmake -DCMAKE_BUILD_TYPE=$ARROW_BUILD_TYPE \
          -DCMAKE_INSTALL_LIBDIR=lib \
          -DCMAKE_INSTALL_PREFIX=$ARROW_HOME \
          -DARROW_PARQUET=on \
          -DARROW_PYTHON=on \
          -DARROW_PLASMA=on \
          -DARROW_BUILD_TESTS=OFF \
          ..
RUN make -j$(nproc)
RUN make install

WORKDIR /arrow/python

RUN python setup.py build_ext --build-type=$ARROW_BUILD_TYPE \
       --with-parquet --inplace
      #--with-plasma  # commented out because plasma tests don't work
RUN py.test pyarrow

来源：https://stackoverflow.com/questions/49059779/how-to-install-pyarrow-on-an-alpine-docker-image

标签

python

Docker

alpine

pyarrow