databricks-connect

How to properly access dbutils in Scala when using Databricks Connect

只愿长相守 提交于 2021-02-07 02:46:44
问题 I'm using Databricks Connect to run code in my Azure Databricks cluster locally from IntelliJ IDEA (Scala). Everything works fine. I can connect, debug, inspect locally in the IDE. I created a Databricks Job to run my custom app JAR, but it fails with the following exception: 19/08/17 19:20:26 ERROR Uncaught throwable from user code: java.lang.NoClassDefFoundError: com/databricks/service/DBUtils$ at Main$.<init>(Main.scala:30) at Main$.<clinit>(Main.scala) Line 30 of my Main.scala class is

DATABRICKS connect 6.4 not able to communicate with server anymore

浪尽此生 提交于 2021-01-28 06:10:19
问题 I am running Pycharm on my MacBook. Client settings: Python Interpreter -> Python 3.7 (dtabricks-connect-6.4) Cluster settings: Databricks Runtime Version -> 6.4 (includes Apache Spark 2.4.5, Scala 2.11) It worked well for months but suddenly, without any updates made, I cant run my python script from Pycharm against databricks cluster anymore. The Error is ... Caused by: `java.lang.IllegalArgumentException: The cluster is running server version `dbr-6.4` but this client only supports Set(dbr