Stop streaming context in Spark Streaming after a period of time

问题

I building an application which receives DStreams from Twitter, the only way to stop the Streaming context is by stoping the execution. I wonder if there is a way to set a time and terminate the streaming socket without stoping the entire application?

回答1:

You can use either

awaitTerminationOrTimeout(long)

as mentioned in the previous answer, or you can stop the streaming context manually from your other thread:

    // in the main thread
    awaitTermination(); // will wait forever or until the context is stopped

    // in another thread
    streamingContext.stop();

回答2:

You can use awaitTermination() method on the streamingContext object to wait for a specified time. Refer this

来源：https://stackoverflow.com/questions/31116817/stop-streaming-context-in-spark-streaming-after-a-period-of-time