问题
How to write Spark Structured Streaming Data into Hive?
There is df.write().saveAsTable(tablename)
however I am not sure if this writes streaming data
I normally do df.writeStream().trigger(new ProcessingTime(1000)).foreach(new KafkaSink()).start()
to write streaming data into Kafka but I don't see anything similar to write streaming data into Hive data warehouse. any ideas?
回答1:
df.createOrReplaceTempView("mytable")
spark.sql("create table mytable as select * from mytable");
or
df.write().mode(SaveMode.Overwrite).saveAsTable("dbName.tableName");
来源:https://stackoverflow.com/questions/45796006/how-to-write-spark-structured-streaming-data-into-hive