tensorflow-serving

So, I've been struggling to understand what the main task of a serving_input_fn() is when a trained model is exported in Tensorflow for serving purposes. There are some examples online that explain it but I'm having problems defining it for myself. The problem I'm trying to solve is a regression problem where I have 29 inputs and one output. Is there a template for creating a corresponding serving input function for that? What if I use a one-class classification problem? Would my serving input function need to change or can I use the same function? And finally, do I always need serving input

I know about the "Serving a Tensorflow Model" page https://www.tensorflow.org/serving/serving_basic but those functions assume you're using tf.Session() which the DNNClassifier tutorial does not... I then looked at the api doc for DNNClassifier and it has an export_savedmodel function (the export function is deprecated) and it seems simple enough but I am getting a "'NoneType' object is not iterable" error... which is suppose to mean I'm passing in an empty variable but I'm unsure what I need to change... I've essentially copied and pasted the code from the get_started/tflearn page on

I would like to include my custom pre-processing logic in my exported Keras model for use in Tensorflow Serving. My pre-processing performs string tokenization and uses an external dictionary to convert each token to an index for input to the Embedding layer: from keras.preprocessing import sequence token_to_idx_dict = ... #read from file # Custom Pythonic pre-processing steps on input_data tokens = [tokenize(s) for s in input_data] token_idxs = [[token_to_idx_dict[t] for t in ts] for ts in tokens] tokens_padded = sequence.pad_sequences(token_idxs, maxlen=maxlen) Model architecture and

I've been following the TensorFlow for Poets 2 codelab on a model I've trained, and have created a frozen, quantized graph with embedded weights. It's captured in a single file - say my_quant_graph.pb . Since I can use that graph for inference with the TensorFlow Android inference library just fine, I thought I could do the same with Cloud ML Engine, but it seems it only works on a SavedModel model. How can I simply convert a frozen/quantized graph in a single pb file to use on ML engine? It turns out that a SavedModel provides some extra info around a saved graph. Assuming a frozen graph

Now we have used TensorFlow to train and export an model. We can implement the inference service with this model just like how tensorflow/serving does. I have a question about whether the tf.Session object is thread-safe or not. If it's true, we may initialize the object after starting and use the singleton object to process the concurrent requests. mrry The tf.Session object is thread-safe for Session.run() calls from multiple threads. Before TensorFlow 0.10 graph modification was not thread-safe. This was fixed in the 0.10 release, so you can add nodes to the graph concurrently with Session