问题
The programming guide mentions that in Spark streaming, to develop custom receivers , it can be done in Java or Scala.
http://spark.apache.org/docs/latest/streaming-custom-receivers.html
However, I am wondering if a custom receiver can also be developed in Python.
Specifically what I am looking for is to receive UDP data stream over a socket in Python. I.e streaming data in UDP format from a device to a mentioned IP address and port number and I want to receive it in Spark streaming. If data is sent via TCP below simple code works fine, however not for UDP.
sc = SparkContext(appName="AppName")
ssc = StreamingContext(sc, 1)
RowsData = ssc.socketTextStream(address, PortNumber)
A similar question was asked by me, but that was for Scala.
Spark Scala UDP receive on listening port
Or is it not possible to create a custom receiver in python or receive UDP packets in Python+Spark_Streaming ?
来源:https://stackoverflow.com/questions/42458812/spark-streaming-custom-receiver-in-python-receive-udp-over-socket