How to generate custom mini-batches using Tensorflow 2.0, such as those in the paper “In defense of the triplet loss”?

给你一囗甜甜゛ 提交于 2021-01-28 07:03:55

问题


I want to implement a custom mini-batch generator in Tensorflow 2.0 using tf.data.Dataset API. Concretely, I have image data, 100 classes with ~200 examples each. For each mini-batch, I want to randomly sample P classes, and K images from each class, for a total of P*K examples in a mini-batch (as described in the paper In Defense of the Triplet Loss for Person Re-Identification]).

I've been searching through documentation for tf.data.Dataset, but can't seem to find the right method. I've looked into the from_generator method, but it doesn't seem suitable for this, since it generates a whole dataset from scratch as I understood.

It seems to me that one way to do it would be to make a new class similar to BatchDataset which can be found in tf.data.Dataset source code, where I would somehow implement the logic, but I'm hoping for an easier solution to be honest.

来源:https://stackoverflow.com/questions/57570385/how-to-generate-custom-mini-batches-using-tensorflow-2-0-such-as-those-in-the-p

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!