Pivotal/Django settings for user provided MSSQL database

Question

I deployed a django application on Pivotal Cloud Foundry. While in development, I just stuck with the built in sqlite database while getting the UI together (didn\'t need to re

Answer 1

Thanks Danial for the post and sharing your code on github. It took me a while to troubleshoot the connection as my installation had a different version of driver. I had ssh-ed into the app to find the driver file and update the version inodbcinst.ini file as below.

[ODBC Driver 17 for SQL Server]
Description=Microsoft ODBC Driver 17 for SQL Server
Driver=/opt/microsoft/msodbcsql17/lib64/libmsodbcsql-17.5.so.2.1
UsageCount=1

Answer 2

ODBC is just a standard framework for connecting to databases. To do anything with it, you need an ODBC driver to talk to your actual database. You are trying to use ODBC, sql_server.pyodbc, but the ODBC driver for SQL Server isn't installed in the application container where your app is running. I'm betting that is a proprietary driver so it just can't be installed by default.

I haven't done this specifically with Python, but I have installed SQL Server ODBC drivers for PHP, and I think it should be roughly the same process (do let me know if this doesn't work).

1. Use the multi-buildpack support in Cloud Foundry. First use the apt-buildpack to install the ODBC drivers. See here for using apt-buildpack. Then use the Python buildpack to actually set up your app.

2. This is the apt.yml file that you'll want. It tells apt-buildpack what to install and it installs the Microsoft packages for their SQL Server ODBC driver and mssql-tools, optional but helpful for validating your initial set up.

   ---
   keys:
   - https://packages.microsoft.com/keys/microsoft.asc
   repos:
   - deb [arch=amd64] https://packages.microsoft.com/ubuntu/18.04/prod bionic main
   packages:
   - msodbcsql17
   - mssql-tools
   - unixodbc-dev
   

3. Then use a manifest.yml file like this to push your app. This will set up the apt-buildpack to run first and install the dependencies above. Then it will run the Python buildpack. It will also bind your MSSQL services, and it sets the env variable ACCEPT_EULA to Y which is required so that the apt-buildpack can install the Microsoft packages without requiring you to manually accept their EULA.

   ---
   applications:
   - name: <app-name>
     buildpacks:
     - https://github.com/cloudfoundry/apt-buildpack
     - python_buildpack
     env:
       ACCEPT_EULA: Y
     services:
     - mssql-server-db
   

4. Add a .profile file with this contents to the root of your project, i.e. from where you run cf push. This puts the mssqltools on the path and also tells unixODBC where to find your ODBC config.

   # the 0 indicates that apt-buildpack is the first in the list
   #  if you change the order of buildpacks you need to update this too
   export PATH=$PATH:/home/vcap/deps/0/apt/opt/mssql-tools/bin/
   
   # point to the odbcinst.ini file
   #  that file needs to have the correct path to the ODBC driver shared library
   #  it should be right, but if you change the order of the buildpacks then that
   #  would need updated too
   export ODBCSYSINI=$HOME/odbc-cfg/
   

5. Lastly, make a folder odbc-cfg (or whatever you want to call it, it just needs to match the path set for ODBCSYSINI in step #4) also in the root of your project. Inside that put the file, odbcinst.ini. Inside that put this info. That will set up the driver.

   [ODBC Driver 17 for SQL Server]
   Description=Microsoft ODBC Driver 17 for SQL Server
   Driver=/opt/microsoft/msodbcsql17/lib64/libmsodbcsql-17.5.so.2.1
   UsageCount=1
   

With that, go ahead and push your app. It should stage and you should see both buildpacks run. Apt buildpack will install the ODBC driver, and Python buildpack will set up your app. Then your app should start and run.

If you have any trouble, run the command cf ssh <app> -t -c "/tmp/lifecycle/launcher /home/vcap/app bash ''". This will put you in the container and have all the env variables source just like for your app. Then run sqlcmd -S <db-host> -U <user> and validate that you can connect to your DB.

---

A few last notes:

• The apt buildpack is going to need internet access to download packages from Ubuntu & Microsoft repositories. This will be needed at staging time. If you don't have internet access from the staging container, you will need to configure a proxy.

• Your container will need to be able to connect to the database. That means the IP has to be routable and no firewalls blocking access. You may also need to have your platform operator add an application security group rule to allow outbound access to your server. If you are unable to connect with sqlcmd, it's likely a network issue so try basic network troubleshooting to make sure you can connect to your server.

• The version of the Microsoft ODBC driver will change. It maybe newer when you read this. The current version reflects the version at the time this update was posted.

Hope that helps!