pip install google-cloud-pubsub fails install in docker container

问题

I am trying to use a pupsub emulator. It starts but when I try to use my python script I get the following error

ModuleNotFoundError: No module named 'google'

So i try to install the module.

RUN pip install google-cloud-pubsub

error

ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3.6 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-2hyoy1ly/grpcio/setup.py'"'"'; __file__='"'"'/tmp/pip-install-2hyoy1ly/grpcio/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-m25l52fe
         cwd: /tmp/pip-install-2hyoy1ly/grpcio/
    Complete output (11 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-2hyoy1ly/grpcio/setup.py", line 191, in <module>
        if check_linker_need_libatomic():
      File "/tmp/pip-install-2hyoy1ly/grpcio/setup.py", line 152, in check_linker_need_libatomic
        stderr=PIPE)
      File "/usr/lib/python3.6/subprocess.py", line 729, in __init__
        restore_signals, start_new_session)
      File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child
        raise child_exception_type(errno_num, err_msg, err_filename)
    FileNotFoundError: [Errno 2] No such file or directory: 'cc': 'cc'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Service 'praise-pubsub' failed to build: The command '/bin/sh -c pip install google-cloud-pubsub==0.24.0' returned a non-zero code: 1

full Dockerfile

FROM google/cloud-sdk:alpine
RUN gcloud components install pubsub-emulator

FROM openjdk:jre-alpine


ENV PYTHONUNBUFFERED=1

RUN echo "**** install Python ****" && \
    apk add --no-cache python3 && \
    if [ ! -e /usr/bin/python ]; then ln -sf python3 /usr/bin/python ; fi && \
    \
    echo "**** install pip ****" && \
    python3 -m ensurepip && \
    rm -r /usr/lib/python*/ensurepip && \
    pip3 install --no-cache --upgrade pip setuptools wheel && \
    if [ ! -e /usr/bin/pip ]; then ln -s pip3 /usr/bin/pip ; fi

#RUN pip install google-cloud <--- still fails when this is here
#RUN pip install Cython --install-option="--no-cython-compile" <--- still fails
RUN pip install google-cloud-pubsub
COPY --from=0 /google-cloud-sdk/platform/pubsub-emulator /pubsub-emulator

回答1:

It looks like in order to install that client, you need gcc installed in your docker container. It is trying to use the cc command to compile part of the library. Try installing the cython package prior to the google-cloud-pubsub package.

It is also worth noting that version 0.24.0 of the Google Cloud Pub/Sub client library is three years old; it is now up to version 1.5.0. This dependency issue (along with many other things) may have been fixed somewhere in the meantime, so it might be worth updating to a more recent version.

回答2:

I'm not sure why you're trying to install python3 & pip3, they both exist in the base image. In any case, this Dockerfile will give you the python google-cloud-pubsub libraries in a nice clean image

FROM google/cloud-sdk:alpine

RUN apk add --no-cache --virtual .build-deps \
    linux-headers build-base g++ python3-dev \
    && pip3 install --no-cache-dir google-cloud-pubsub \
    && apk del .build-deps

# add your stuff here

回答3:

If you look at the downloadable files for grpcio, the problem package, you'll see that there are precompiled binary wheels:

https://pypi.org/project/grpcio/#files

So why is this build trying to compile from scratch? Because you're using Alpine. Alpine doesn't support binary wheels (see https://pythonspeed.com/articles/alpine-docker-python/ for longer explanation).

As others say, you can install a compiler... or you can just stop using an Alpine-based Docker image, and then you will be able to use precompiled wheels, and your builds will be faster. And your images will also be smaller since you won't need to install a compiler (you can make Alpine images smaller wiht multi-stage builds, but that's more work).

来源：https://stackoverflow.com/questions/62260186/pip-install-google-cloud-pubsub-fails-install-in-docker-container

标签

python

Docker

google-cloud-pubsub