Is it possible to write Flume headers to HDFS sink and drop the body?

前端 未结 1 2018
一个人的身影
一个人的身影 2021-01-23 04:02

The text_with_headers serializer (HDFS sink serializer) allows to save the Flume event headers rather than discarding them. The output format consists of the headers, followed b

1条回答
  •  一向
    一向 (楼主)
    2021-01-23 04:08

    You can set serializer property to header_and_text, which outputs both the headers and the body.

    For example:

    agent.sinks.my-hdfs-sink.type = hdfs
    agent.sinks.my-hdfs-sink.hdfs.fileType = DataStream
    ... 
    # very important
    agent.sinks.my-hdfs-sink.serializer = header_and_text
    

    0 讨论(0)
提交回复
热议问题