Spark Streaming custom receiver in “Python” (receive UDP over socket)

删除回忆录丶 提交于 2019-12-12 03:55:49

问题


The programming guide mentions that in Spark streaming, to develop custom receivers , it can be done in Java or Scala.
http://spark.apache.org/docs/latest/streaming-custom-receivers.html
However, I am wondering if a custom receiver can also be developed in Python.

Specifically what I am looking for is to receive UDP data stream over a socket in Python. I.e streaming data in UDP format from a device to a mentioned IP address and port number and I want to receive it in Spark streaming. If data is sent via TCP below simple code works fine, however not for UDP.

sc = SparkContext(appName="AppName")
ssc = StreamingContext(sc, 1)
RowsData = ssc.socketTextStream(address, PortNumber)

A similar question was asked by me, but that was for Scala.
Spark Scala UDP receive on listening port

Or is it not possible to create a custom receiver in python or receive UDP packets in Python+Spark_Streaming ?

来源:https://stackoverflow.com/questions/42458812/spark-streaming-custom-receiver-in-python-receive-udp-over-socket

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!