tfrecord

问题 Suppose I have 3 tfrecord files, namely neg.tfrecord , pos1.tfrecord , pos2.tfrecord . I use dataset = tf.data.TFRecordDataset(tfrecord_file) this code creates 3 Dataset objects. My batch size is 400, including 200 neg data, 100 pos1 data, and 100 pos2 data. How can I get the desired dataset? I will use this dataset object in keras.fit() (Eager Execution). My tensorflow's version is 1.13.1. Before, I tried to get the iterator for each dataset, and then manually concat after getting the data,

问题 Considering that is possible to run tf.Data.Datasets in eager execution mode, how should I open a TFRecord file on eager execution? I'm more concerned about the parser writing, because I'm currently using dataset.make_one_shot_iterator as an iterator (between several images on my container). 回答1: In TensorFlow 1.8 onwards you can naturally iterate on the tf.data.Dataset object with eager execution enabled. ds = tf.data.TFRecordDataset(...).map(...).batch(...) for x in ds: print(x) make_one

问题 I am trying to use the 128 byte embeddings produced by the pre-trained base of the VGGish model for transfer learning on audio data. Using python vggish_inference_demo.py --wav_file ... to encode my training data to a tfrecord worked fine, but now I want to use this as an input to another model (e.g. a neural network I create with keras or something else). Using some similar questions and the documentation, I go this far with the first embedding record of one file: tfrecords_filename =

问题 I have 5 tfrecords files, one for each object. While training I want to read data equally from all the 5 tfrecords i.e. if my batch size is 50, I should get 10 samples from 1st tfrecord file, 10 samples from the second tfrecord file and so on. Currently, it just reads sequentially from all the three files i.e. I get 50 samples from the same record. Is there a way to sample from differnt tfrecords files? 回答1: I advise you to read the tutorial by @mrry on tf.data . On slide 42 he explains how

问题 This is my code! My tensorflow version is 1.6.0, python version is 3.6.4. If I direct use dataset to read csv file, I can train and no wrong. But I convert csv file to tfrecords file, it's wrong. I google it in Internet and almost people say tensorflow should be updated, but it don't work for me. import tensorflow as tf tf.logging.set_verbosity(tf.logging.INFO) feature_names = [ 'SepalLength', 'SepalWidth', 'PetalLength', 'PetalWidth' ] def my_input_fn(is_shuffle=False, repeat_count=1):

问题 When I read file with tf.read_file I get something with type tf.string . Documentation says only that it is "Variable length byte arrays. Each element of a Tensor is a byte array." (https://www.tensorflow.org/versions/r0.10/resources/dims_types.html). I have no idea how to interpret this. I can do nothing with this type. In usual python you can get elements by index like my_string[:4] , but when I run following code I get an error. import tensorflow as tf import numpy as np x = tf.constant(

问题 I have a .tfrecord but I don't know how it is structured. How can I inspect the schema to understand what the .tfrecord file contains? All Stackoverflow answers or documentation seem to assume I know the structure of the file. reader = tf.TFRecordReader() file = tf.train.string_input_producer("record.tfrecord") _, serialized_record = reader.read(file) ...HOW TO INSPECT serialized_record... 回答1: Found it! import tensorflow as tf for example in tf.python_io.tf_record_iterator("data/foobar

问题 My question is about how to get batch inputs from multiple (or sharded) tfrecords. I've read the example https://github.com/tensorflow/models/blob/master/inception/inception/image_processing.py#L410. The basic pipeline is, take the training set as as example, (1) first generate a series of tfrecords (e.g., train-000-of-005 , train-001-of-005 , ...), (2) from these filenames, generate a list and fed them into the tf.train.string_input_producer to get a queue, (3) simultaneously generate a tf

问题 My question is about how to get batch inputs from multiple (or sharded) tfrecords. I've read the example https://github.com/tensorflow/models/blob/master/inception/inception/image_processing.py#L410. The basic pipeline is, take the training set as as example, (1) first generate a series of tfrecords (e.g., train-000-of-005 , train-001-of-005 , ...), (2) from these filenames, generate a list and fed them into the tf.train.string_input_producer to get a queue, (3) simultaneously generate a tf

问题 I use Python 2.7.13 and Tensorflow 1.3.0 on CPU. I want to use DensNet( https://github.com/pudae/tensorflow-densenet ) for regression problem. My data contains 60000 jpeg images with 37 float labels for each image. I saved my data into tfrecords files by: def Read_Labels(label_path): labels_csv = pd.read_csv(label_path) labels = np.array(labels_csv) return labels[:,1:] ` def load_image(addr): # read an image and resize to (224, 224) img = cv2.imread(addr) img = cv2.resize(img, (224, 224),