com.mysql.jdbc.Driver not found in spark2 scala

此生再无相见时 提交于 2019-12-11 11:46:56

问题


I am using Jupyter Notebook with Scala kernel, below is my code to import mysql table to a dataframe:

val sql="""select * from customer"""
val df_customer = spark.read
  .format("jdbc")
  .option("url", "jdbc:mysql://localhost:3306/ccfd")
  .option("driver", "com.mysql.jdbc.Driver")
  .option("dbtable",  s"( $sql ) t")
  .option("user", "root")
  .option("password", "xxxxxxx")
  .load()

Below is the error:

Name: java.lang.ClassNotFoundException
Message: com.mysql.jdbc.Driver
StackTrace:   at scala.reflect.internal.util.AbstractFileClassLoader.findClass(AbstractFileClassLoader.scala:62)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
  at org.apache.spark.sql.execution.datasources.jdbc.DriverRegistry$.register(DriverRegistry.scala:45)
  at org.apache.spark.sql.execution.datasources.jdbc.JDBCOptions$$anonfun$6.apply(JDBCOptions.scala:79)
  at org.apache.spark.sql.execution.datasources.jdbc.JDBCOptions$$anonfun$6.apply(JDBCOptions.scala:79)
  at scala.Option.foreach(Option.scala:257)
  at org.apache.spark.sql.execution.datasources.jdbc.JDBCOptions.<init>(JDBCOptions.scala:79)
  at org.apache.spark.sql.execution.datasources.jdbc.JDBCOptions.<init>(JDBCOptions.scala:35)
  at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:34)
  at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:340)
  at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:239)
  at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:227)
  at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:164)

Can anyone share a working code snippet here? I am using Spark2, session named spark is ready when I start the kernel in a new notebook.

Thank you in advance.

来源:https://stackoverflow.com/questions/55154372/com-mysql-jdbc-driver-not-found-in-spark2-scala

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!