Keras shows no Improvements to training speed with GPU (partial GPU usage?!)

Question

I am trying to train my model on a GPU instead of a CPU on an AWS p2.xlarge instance from my Jupyter Notebook. I am using tensorflow-gpu backend (only tensorflow-gpu

Answer 1

That happens because you're using LSTM layers.

Tensorflow's implementation for LSTM layers is not that great. The reason is probably that recurrent calculations are not parallel calculations, and GPUs are great for parallel processing.

I confirmed that by my own experience:

• Got terrible speed using LSTMs in my model
• Decided to test the model removing all LSTMs (got a pure convolutional model)
• The resulting speed was simply astonishing!!!

This article about using GPUs and tensorflow also confirms that:

• http://minimaxir.com/2017/07/cpu-or-gpu/

A possible solution?

You may try using the new CuDNNLSTM, which seems prepared specially for using GPUs.

I never tested it, but you'll most probably get a better performance with this.

Another thing that I haven't tested, and I'm not sure it's designed for that reason, but I suspect it is: you can put unroll=True in your LSTM layers. With that, I suspect the recurrent calculations will be transformed in parallel ones.