Parallel training on multiple GPUs, Can not boost GPU-Util

Question

I am using Tensorflow input data pipeline to read data from a big csv file and MirroredStrategy to train a neural network on multiple GPUs on same cluster. My problem is tha