Could Keras prefetch data like tensorflow Dataset?

痞子三分冷 提交于 2021-01-27 01:39:10

问题


In TensorFlow's Dataset API, we can use dataset.prefetch(buffer_size=xxx) to preload other batches' data while GPU is processing the current batch's data, therefore, I can make full use of GPU.

I'm going to use Keras, and wonder if keras has a similar API for me to make full use of GPU, instead of serial execution: read batch 0->process batch 0->read batch 1-> process batch 1-> ...

I briefly looked through the keras API and did not see a description of the prefetch.


回答1:


If you call fit_generator with workers > 1, use_multiprocessing=True, it will prefetch queue_size batches.

From docs: max_queue_size: Integer. Maximum size for the generator queue. If unspecified, max_queue_size will default to 10.



来源:https://stackoverflow.com/questions/52176792/could-keras-prefetch-data-like-tensorflow-dataset

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!