tensorflow GPU crashes for 0 batch size CUDNN_STATUS_BAD_PARAM

Question

This issue seem to be existing for a long time and lots of users are facing the issue.

stream_executor/cuda/cuda_dnn.cc:444] could not convert BatchDescriptor {count: 0

Answer 1

The problem is when you model received 0 batch size. For me I had the error because I have 1000 example and I run it on multiple GPus ( 2 GPU) with batch size equal to 32 .And in My graph I divided the batch size to mini batch size to so each GPU take 16 example. At step 31 ( 31 * 32) I will finished 992 examples , so there is only 8 example left, it will go to GPU 1 and GPU2 will end with zero batch size that's why I received your error above.

Still couldn't solve it and still searching about proper solution. I hope this help you to discover when in your code you received zero batch size.