I have a single source CSV file containing records of different sizes that pushes every record into one source topic. I want to split the records into different KStreams/KTables
I am able to split the data and used KSQL for the approach that I am sharing below.
1. An input stream is created with value_format='JSON'
and a column payload
as STRING
2. The payload will contain the whole record as a STRING
3. The record is then split into different streams using LIKE
operator in the WHERE
clause while putting the payload into different streams as per the requirement. Here I have used SPLIT
operator of KSQL to get the records from payload that are in comma-delimited format