tensorflow-serving

问题 I was trying to do predictions on gcloud ml-engine with the tensorflow object detection pets example, but it doesn't work. I created a checkpoint using this example: https://github.com/tensorflow/models/blob/master/object_detection/g3doc/running_pets.md With the help of the tensorflow team, I was able to create an saved_model to upload to the gcloud ml-engine: https://github.com/tensorflow/models/issues/1811 Now, I can upload the model to the gcloud ml-engine. But unfortunately, I'm not able

问题 I'm busy configuring a TensorFlow Serving client that asks a TensorFlow Serving server to produce predictions on a given input image, for a given model. If the model being requested has not yet been served, it is downloaded from a remote URL to a folder where the server's models are located. (The client does this). At this point I need to update the model_config and trigger the server to reload it. This functionality appears to exist (based on https://github.com/tensorflow/serving/pull/885

问题 So, I've been struggling to understand what the main task of a serving_input_fn() is when a trained model is exported in Tensorflow for serving purposes. There are some examples online that explain it but I'm having problems defining it for myself. The problem I'm trying to solve is a regression problem where I have 29 inputs and one output. Is there a template for creating a corresponding serving input function for that? What if I use a one-class classification problem? Would my serving

问题 Now we have used TensorFlow to train and export an model. We can implement the inference service with this model just like how tensorflow/serving does. I have a question about whether the tf.Session object is thread-safe or not. If it's true, we may initialize the object after starting and use the singleton object to process the concurrent requests. 回答1: The tf.Session object is thread-safe for Session.run() calls from multiple threads. Before TensorFlow 0.10 graph modification was not thread

问题 I am trying to deploy a tf.keras image classification model to Google CloudML Engine. Do I have to include code to create serving graph separately from training to get it to serve my models in a web app? I already have my model in SavedModel format ( saved_model.pb & variable files), so I'm not sure if I need to do this extra step to get it to work. e.g. this is code directly from GCP Tensorflow Deploying models documentation def json_serving_input_fn(): """Build the serving inputs.""" inputs

问题 Does anybody know how to create a C# client for tensorflow serving? My tensorflow serving installation: I installed tensorflow serving using the tensorflow serving dockerfile, then inside the container I did the following: pip install tensorflow pip install tensorflow-serving-api echo "deb [arch=amd64] http://storage.googleapis.com/tensorflow-serving-apt stable tensorflow-model-server tensorflow-model-server-universal" | tee /etc/apt/sources.list.d/tensorflow-serving.list curl https://storage

问题 I trained a Wide & Deep model using the pre-made Estimator class (DNNLinearCombinedClassifier), by essentially following the tutorial on tensorflow.org. I wanted to do inference/serving, but without using tensorflow-serving. This basically comes down to feeding some test data to the correct input tensor and retrieving the output tensor. However, I am not sure what the input nodes/layer should be. In the tensorflow graph (graph.pbtxt), the following nodes seem relevant. But they are also

问题 Overview I followed the following guide to write TF Records, where I used tf.Transform to preprocess my features. Now, I would like to deploy my model, for which I need apply this preprocessing function on real live data. My Approach First, suppose I have 2 features: features = ['amount', 'age'] I have the transform_fn from the Apache Beam, residing in working_dir=gs://path-to-transform-fn/ Then I load the transform function using: tf_transform_output = tft.TFTransformOutput(working_dir) I

问题 Having seen this github issue and this stackoverflow post I had hoped this would simply work. It seems as though passing in the environment variable MODEL_CONFIG_FILE has no affect. I am running this through docker-compose but I get the same issue using docker-run . The error: I tensorflow_serving/model_servers/server.cc:82] Building single TensorFlow model file config: model_name: model model_base_path: /models/model I tensorflow_serving/model_servers/server_core.cc:461] Adding/updating

问题 I create a lookup table from tf.contrib.lookup, using the training data (as input). Then, I pass every input through that lookup table, before passing it through my model. This works for training, but when it comes to online prediction from this same model, it raises the error: Table not initialized I'm using SavedModel to save the model. I run the prediction from this saved model. How can I initialize this table so that it stays initialized? Or is there a better way to save the model so that