Converting CSV to JSON using pyspark with filter and additional columns

Question

Given a CSV Input file with Headers: "CorrelationID", "Message", "EventTimeStamp", "Flag", "RandomColumns"

12345, &q