tensorflow-datasets

来源： https://stackoverflow.com/questions/56426839/why-does-tf-data-dataset-map-run-only-once

来源： https://stackoverflow.com/questions/63092118/valueerror-input-0-of-layer-sequential-9-is-incompatible-with-the-layer-expect

问题 I read in a *.csv file using tf.data.TextLineDataset and apply map on it: dataset = tf.data.TextLineDataset(os.path.join(data_dir, subset, 'label.txt')) dataset = dataset.map(lambda value: parse_record_fn(value, is_training), num_parallel_calls=num_parallel_calls) Parse function parse_record_fn looks like this: def parse_record(raw_record, is_training): default_record = ["./", -1] filename, label = tf.decode_csv([raw_record], default_record) # do something return image, label But there raise

问题 I am new in tensoflow and I want to adapt the MNIST tutorial https://www.tensorflow.org/tutorials/layers with my own data (images of 40x40). This is my model function : def cnn_model_fn(features, labels, mode): # Input Layer input_layer = tf.reshape(features, [-1, 40, 40, 1]) # Convolutional Layer #1 conv1 = tf.layers.conv2d( inputs=input_layer, filters=32, kernel_size=[5, 5], # To specify that the output tensor should have the same width and height values as the input tensor # value can be

问题 I am new in tensoflow and I want to adapt the MNIST tutorial https://www.tensorflow.org/tutorials/layers with my own data (images of 40x40). This is my model function : def cnn_model_fn(features, labels, mode): # Input Layer input_layer = tf.reshape(features, [-1, 40, 40, 1]) # Convolutional Layer #1 conv1 = tf.layers.conv2d( inputs=input_layer, filters=32, kernel_size=[5, 5], # To specify that the output tensor should have the same width and height values as the input tensor # value can be

问题 I have tokenized data in the form of a list of unequally shaped arrays: array([array([1179, 6, 208, 2, 1625, 92, 9, 3870, 3, 2136, 435, 5, 2453, 2180, 44, 1, 226, 166, 3, 4409, 49, 6728, ... 10, 17, 1396, 106, 8002, 7968, 111, 33, 1130, 60, 181, 7988, 7974, 7970])], dtype=object) With their respective targets: Out[74]: array([0, 0, 0, ..., 0, 0, 1], dtype=object) I'm trying to transform them into a padded tf.data.Dataset() , but it won't let me convert unequal shapes to a tensor. I will get

问题 I have tokenized data in the form of a list of unequally shaped arrays: array([array([1179, 6, 208, 2, 1625, 92, 9, 3870, 3, 2136, 435, 5, 2453, 2180, 44, 1, 226, 166, 3, 4409, 49, 6728, ... 10, 17, 1396, 106, 8002, 7968, 111, 33, 1130, 60, 181, 7988, 7974, 7970])], dtype=object) With their respective targets: Out[74]: array([0, 0, 0, ..., 0, 0, 1], dtype=object) I'm trying to transform them into a padded tf.data.Dataset() , but it won't let me convert unequal shapes to a tensor. I will get

问题 After reading through the tf.data documentation (here for TF 1.15), related TF code (both Python and C++), I realized that most of it seems to run purely on CPU, except of PrefetchDataset . Is that true? The documentation for prefetch_to_device says: NOTE: Although the transformation creates a tf.data.Dataset , the transformation must be the final Dataset in the input pipeline. Which suggest that all other datasets cannot handle such a GPU-based dataset. While looking through the code, there

问题 I'd like to speed up my training routine that uses the Estimator API with input_fn wrote using tf.data.Dataset . My implementation takes 2 second to prepare a batch of data and then runs training on GPU for 1 sec, and then start over preparing a batch. Which is really inefficient. I'm looking for a way to prepare the batches asynchronously and upload them to GPU to speed up the training. Or alternatively for a way to cache datasets between invocations of input_fn (the dataset.cache() doesn't

问题 I'd like to speed up my training routine that uses the Estimator API with input_fn wrote using tf.data.Dataset . My implementation takes 2 second to prepare a batch of data and then runs training on GPU for 1 sec, and then start over preparing a batch. Which is really inefficient. I'm looking for a way to prepare the batches asynchronously and upload them to GPU to speed up the training. Or alternatively for a way to cache datasets between invocations of input_fn (the dataset.cache() doesn't