问题
In TensorFlow's Dataset API, we can use dataset.prefetch(buffer_size=xxx)
to preload other batches' data while GPU is processing the current batch's data, therefore, I can make full use of GPU.
I'm going to use Keras, and wonder if keras
has a similar API for me to make full use of GPU, instead of serial execution: read batch 0->process batch 0->read batch 1-> process batch 1-> ...
I briefly looked through the keras
API and did not see a description of the prefetch.
回答1:
If you call fit_generator
with workers
> 1, use_multiprocessing=True
, it will prefetch queue_size batches.
From docs: max_queue_size: Integer. Maximum size for the generator queue. If unspecified, max_queue_size will default to 10.
来源:https://stackoverflow.com/questions/52176792/could-keras-prefetch-data-like-tensorflow-dataset