Google Composer- How do I install Microsoft SQL Server ODBC drivers on environments

Question

I am new to GCP and Airflow and am trying to run my python pipelines via a simple PYODBC connection via python 3. However, I believe I have found what I need to install on t

Answer 1

Cloud Composer currently primarily supports installing PyPI packages written in pure Python. Installing system packages is not fully supported at this time, but there are some workarounds (such as setting LD_LIBRARY_PATH and uploading shared libraries, etc). You're getting aborts because you installed the Python part of the package, but not system dependencies the Python package depends on.

As you read, changes to Airflow workers in Composer are ephemeral (or at least, should be treated as such), but one way of working around this is to install packages using BashOperator before the task that needs the library runs. It's not pretty, but it ensure that dependencies are installed on the worker before the Python code that needs them is called.