How to make a confusion matrix with a keras model and TFrecords data?

Question

I have trained a keras model which uses TFrecords because the training data and also validation data does not fit into memory. Now I would like to do a confusion matrix but the