Connect docker python to SQL server with pyodbc

Question

I\'m trying to connect a pyodbc python script running in a docker container to login to a MSSQL database I have tried all sorts of docker files, but not been able to make th

Answer 1

I was unable to use all of the above resolutions, I was keeping al kind of errors relating to the pyodbc package, in particular:

ImportError: libodbc.so.2: cannot open shared object file: No such file or directory.

I ended up with another resolution which defines the ODBC SQL Server Driver specifically for an Ubuntu 18.04 Docker image, in this case ODBC Driver 17 for SQL Server. In my specific use case I needed to make the connection to my MySQL database server on Azure via Flask SQLAlchemy, but the latter is not a necessity for the Docker configuration.

Dockerfile, with most important part adding the Microsoft repository and installing msodbcsql17 and unixodbc-dev:

# Ubuntu 18.04 base with Python runtime and pyodbc to connect to SQL Server
FROM ubuntu:18.04

WORKDIR /app

# apt-get and system utilities
RUN apt-get update && apt-get install -y \
    curl apt-utils apt-transport-https debconf-utils gcc build-essential g++-5\
    && rm -rf /var/lib/apt/lists/*

# adding custom Microsoft repository
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN curl https://packages.microsoft.com/config/ubuntu/18.04/prod.list > /etc/apt/sources.list.d/mssql-release.list

# install SQL Server drivers
RUN apt-get update && ACCEPT_EULA=Y apt-get install -y msodbcsql17 unixodbc-dev

# install SQL Server tools
RUN apt-get update && ACCEPT_EULA=Y apt-get install -y mssql-tools
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc
RUN /bin/bash -c "source ~/.bashrc"

# python libraries
RUN apt-get update -y && \
    apt-get install -y python3-pip python3-dev

# install necessary locales, this prevents any locale errors related to Microsoft packages
RUN apt-get update && apt-get install -y locales \
    && echo "en_US.UTF-8 UTF-8" > /etc/locale.gen \
    && locale-gen

# copy requirements and install packages, I added this for general use
COPY ./requirements.txt > ./requirements.txt
RUN pip3 install -r ./requirements.txt
# you can also use regular install of the packages
RUN pip3 install pyodbc SQLAlchemy

# and if you are also planning to use Flask and Flask-SQLAlchemy
Run pip3 install Flask Flask-SQLAlchemy

COPY ..

# run your app via entrypoint or change the CMD command to your regular command
COPY docker-entrypoint.sh wsgi.py ./
CMD ["./docker-entrypoint.sh"]

This should build without any errors in Docker.

My database url looked like this:

import urllib.parse

# name the sepcific ODBC driver by version number, we installed msodbcsql17
params = urllib.parse.quote_plus("DRIVER={ODBC Driver 17 for SQL Server};SERVER=<your.database.windows.net>;DATABASE=<your-db-name>;UID=<username>;PWD=<password>")

db_uri = "mssql+pyodbc:///?odbc_connect={PARAMS}".format(PARAMS=params)

And for the bonus if you are using Flask-SQLAlchemy, your app config should contain something like this:

app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
app.config["SQLALCHEMY_DATABASE_URI"] = db_uri  # from above

Happy coding!

Answer 2

I fixed this problem by using pypyodbc instead of pyodbc.

pip install pypyodbc==1.3.5

https://pypi.org/project/pypyodbc/

Found the hint here: https://github.com/Azure/azure-functions-python-worker/issues/249

Answer 3

Based on Kåre Rasmussen's answer, here's a complete dockerfile for further use.

Make sure to edit the last two lines according to your architecture! They should reflect the actual paths to libtdsodbc.so and libtdsS.so.

If you're not sure about the paths to libtdsodbc.so and libtdsS.so, try dpkg --search libtdsodbc.so and dpkg --search libtdsS.so.

FROM python:3

#Install FreeTDS and dependencies for PyODBC
RUN apt-get update && apt-get install -y tdsodbc unixodbc-dev \
 && apt install unixodbc-bin -y  \
 && apt-get clean -y

RUN echo "[FreeTDS]\n\
Description = FreeTDS unixODBC Driver\n\
Driver = /usr/lib/arm-linux-gnueabi/odbc/libtdsodbc.so\n\
Setup = /usr/lib/arm-linux-gnueabi/odbc/libtdsS.so" >> /etc/odbcinst.ini

Afterwards, install PyODBC, COPY your app and run it.

Answer 4

For me to solve this issue I also had to add the following 2 lines in the dockerfile:

RUN echo MinProtocol = TLSv1.0 >> /etc/ssl/openssl.cnf
RUN echo CipherString = DEFAULT@SECLEVEL=1 >> /etc/ssl/openssl.cnf

Answer 5

How to install the necessary dependencies for pyodbc is related to the linux distribution and its version (in docker case, that is the base image of your docker image). If none of the above work for you, you can figure out the commands by trying in the docker container instance.

First, exec into the docker container

docker exec -it <container id> bash

Try various ways to get the distribution name and version of your linux. Then try different instructions in Install the Microsoft ODBC driver for SQL Server (Linux)

Here is a working example for Debian 9 based images, deriving exactly as the document instructions.

# Install pyodbc dependencies
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN curl https://packages.microsoft.com/config/debian/9/prod.list > /etc/apt/sources.list.d/mssql-release.list
RUN apt-get update
RUN ACCEPT_EULA=Y apt-get -y install msodbcsql17
RUN apt-get -y install unixodbc-dev
RUN pip install pyodbc

Answer 6

Running through this recently I found it was necessary to additionally include the following line (note that it did not build without this step):

RUN apt-get install --reinstall build-essential -y

The full Dockerfile looks as follows:

# parent image
FROM python:3.7-slim

# install FreeTDS and dependencies
RUN apt-get update \
 && apt-get install unixodbc -y \
 && apt-get install unixodbc-dev -y \
 && apt-get install freetds-dev -y \
 && apt-get install freetds-bin -y \
 && apt-get install tdsodbc -y \
 && apt-get install --reinstall build-essential -y

# populate "ocbcinst.ini"
RUN echo "[FreeTDS]\n\
Description = FreeTDS unixODBC Driver\n\
Driver = /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so\n\
Setup = /usr/lib/x86_64-linux-gnu/odbc/libtdsS.so" >> /etc/odbcinst.ini

# install pyodbc (and, optionally, sqlalchemy)
RUN pip install --trusted-host pypi.python.org pyodbc==4.0.26 sqlalchemy==1.3.5

# run app.py upon container launch
CMD ["python", "app.py"]

Here's one way to then actually establish the connection inside app.py, via sqlalchemy (and assuming port 1433):

import sqlalchemy as sa
args = (username, password, server, database)
connstr = "mssql+pyodbc://{}:{}@{}/{}?driver=FreeTDS&port=1433&odbc_options='TDS_Version=8.0'"
engine = sa.create_engine(connstr.format(*args))