hortonworks-dataflow

问题 I am testing out NiFi to replace our current ingestion setup which imports data from multiple MySQL shards of a table and store it in HDFS. I am using GenerateTableFetch and ExecuteSQL to achieve this. Each incoming flow file will have a database.name attribute which is being used by DBCPConnectionPoolLookup to select the relevant shard. Issue is that, let's say I have 2 shards to pull data from, shard_1 and shard_2 for table accounts and also I have updated_at as Maximum Value Columns , it

问题 Following are the combination of processors that I am using:- GetFile + SplitText + ExtractText + UpdateAttribute + ExecuteSQL + ConvertAvroToJson + PutFile Basically,I have a properties file which contains 5 comma separated values that are required by the 'DBCPConnectionPool' controller service to establish connection with the database. Here is the content of my properties file:- jdbc:mysql://localhost:3306/test,com.mysql.jdbc.Driver,C:\Program Files\MySQL\mysql-connector.jar,root,root Now,

问题 I want to convert JSON files to CSV in nifi. We can achieve this in Python and other programming languages and have multiple articles on it. I have multiple JSON files and each file has different schema(one specific file will have one schema only). I can see there are templates to convert CSV to JSON and other conversions. But I didn't see any template to convert JSON data to CSV. I have gone through the article https://community.hortonworks.com/articles/64069/converting-a-large-json-file