Do GPUs have a noticeable improvement in ML predictions? (not training)

Question

I interested in improving performance with regard to ML predictions. (I don\'t care about training) -Will GPUs provide more throughput or lower latency? -Are they good for batch