How to reduce tensorflow dataset input pipeline host device (cpu) time (currently ~40%)?

Question

I am trying to replicate the resnet18 paper. Before running this on the full Image Net dataset on disk, I\'m doing some evaluation runs with the publicly available imagene