nifi ConvertRecord JSON to CSV getting only single record?

Question

I have the below flow set up for reading json data and convert it to csv using the convertRecord processor. However, the output flowfile is only populated with single re

Answer 1

You are facing this NIFI-4456 bug and has been fixed starting from NiFi-1.7 version.

To work around this issue:

1.use SplitText processor with split line count =1

2.Then use MergeContent/MergeRecord processors (using Defragments as merge strategy) and make as valid json array of messages

if you are using Merge Record processor then Reader and Writer controller services needs to be in Json format.

3.Then feed the merged relation to ConvertRecord processor.

Flow:

Starting from NiFi-1.7+ version, we don't need to configure any thing new/additional in JsonTreeReader controller service as NiFi is able to read json per line format also.

Update:

MergeContent Configs:

if we are using MergeContent processor please configure the processor like this way as shown in the below screenshot.

Delimiter Strategy Text

Header [

Footer ]

Demarcator ,

In addition i would suggest to use MergeRecord processor instead of MergeContent processor which will take care of creating a valid array of json messages.