Low GPU usage by Keras / Tensorflow?

Question

I\'m using keras with tensorflow backend on a computer with a nvidia Tesla K20c GPU. (CUDA 8)

I\'m tranining a relatively simple Convolutional Neural Network, during tra

Answer 1

You should find the bottleneck:

On windows use Task-Manager> Performance to monitor how you are using your resources

On Linux use nmon, nvidia-smi, and htop to monitor your resources.

The most possible scenarios are:

• If you have a huge dataset, take a look at the disk read/write rates; if you are accessing your hard-disk frequently, most probably you need to change they way you are dealing with the dataset to reduce number of disk access

• Use the memory to pre-load everything as much as possible.

• If you are using a restful API or any similar services, make sure that you do not wait much for receiving what you need. For restful services, the number of requests per second might be limited (check your network usage via nmon/Task manager)

• Make sure you do not use swap space in any case!

• Reduce the overhead of preprocessing by any means (e.g. using cache, faster libraries, etc.)

• Play with the bach_size (however, it is said that higher values (>512) for batch size might have negative effects on accuracy)