Databricks (Spark): .egg dependencies not installed automatically?

Question

I have a locally created .egg package that depends on boto==2.38.0. I used setuptools to create the build distribution. Everything works in my own loca

Answer 1

Your application's dependencies will not, in general, work properly if they are diverse and don't have uniform language support. The Databrick docs explain that

Databricks will install the correct version if the library supports both Python 2 and 3. If the library does not support Python 3 then library attachment will fail with an error.

In this case it will not automatically fetch dependencies when you attach a library to the cluster.