Convert JSON to CSV in nifi

廉价感情. 提交于 2019-12-10 21:49:08

问题


I want to convert JSON files to CSV in nifi. We can achieve this in Python and other programming languages and have multiple articles on it. I have multiple JSON files and each file has different schema(one specific file will have one schema only). I can see there are templates to convert CSV to JSON and other conversions. But I didn't see any template to convert JSON data to CSV. I have gone through the article https://community.hortonworks.com/articles/64069/converting-a-large-json-file-into-csv.html ,however here we are hard coding the schema. As I have multiple files and each file has different schema, I can't hardcode the schema. Any suggestions please.


回答1:


Conversion between formats is typically done through ConvertRecord by plugging in the appropriate record reader and record writer, in this case a JSON reader and CSV writer.

To make use of the record processors you need to defined Avro schemas for your data and put them in a schema registry, NiFi provides a local one.

There are lots of examples and posts out there about the record stuff, this slide deck shows an example of CSV to JSON, but would be easy to reverse the situation for your scenario:

https://www.slideshare.net/BryanBende/apache-nifi-record-processing

This post has some other info: https://bryanbende.com/development/2017/06/20/apache-nifi-records-and-schema-registries



来源:https://stackoverflow.com/questions/49145832/convert-json-to-csv-in-nifi

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!