pip install google-cloud-pubsub fails install in docker container

Question

I am trying to use a pupsub emulator. It starts but when I try to use my python script I get the following error

ModuleNotFoundError: No module named \'google\'
         


        

Answer 1

It looks like in order to install that client, you need gcc installed in your docker container. It is trying to use the cc command to compile part of the library. Try installing the cython package prior to the google-cloud-pubsub package.

It is also worth noting that version 0.24.0 of the Google Cloud Pub/Sub client library is three years old; it is now up to version 1.5.0. This dependency issue (along with many other things) may have been fixed somewhere in the meantime, so it might be worth updating to a more recent version.

Answer 2

I'm not sure why you're trying to install python3 & pip3, they both exist in the base image. In any case, this Dockerfile will give you the python google-cloud-pubsub libraries in a nice clean image

FROM google/cloud-sdk:alpine

RUN apk add --no-cache --virtual .build-deps \
    linux-headers build-base g++ python3-dev \
    && pip3 install --no-cache-dir google-cloud-pubsub \
    && apk del .build-deps

# add your stuff here

Answer 3

If you look at the downloadable files for grpcio, the problem package, you'll see that there are precompiled binary wheels:

https://pypi.org/project/grpcio/#files

So why is this build trying to compile from scratch? Because you're using Alpine. Alpine doesn't support binary wheels (see https://pythonspeed.com/articles/alpine-docker-python/ for longer explanation).

As others say, you can install a compiler... or you can just stop using an Alpine-based Docker image, and then you will be able to use precompiled wheels, and your builds will be faster. And your images will also be smaller since you won't need to install a compiler (you can make Alpine images smaller wiht multi-stage builds, but that's more work).