Are shared variables supported in Spark Streaming?

问题

Spark provide two limited types of shared variables for two common usage patterns: broadcast variables and accumulators, are they supported in Spark Streaming??

回答1:

Yes, you can use them as you'd normally do via SparkContext. The only difference is that you need to get it from SparkStreamingContext:

val sparkConf = new SparkConf().setAppName("MyApp")
val ssc = new StreamingContext(sparkConf, Seconds(1))
ssc.sparkContext.broadcast(myValue)

With spark streaming you might want to update that value at some point, to do that you should use unpersist() (or the blocking version) and broadcast again.

来源：https://stackoverflow.com/questions/35097953/are-shared-variables-supported-in-spark-streaming