I followed the official tutorial on how to create/load TFRecords here.
So, now my dataset contains examples in the form of {"feat1":..., "feat2"
{"feat1":..., "feat2"