tensorflow-serving

Graph optimizations on a tensorflow serveable created using tf.Estimator

由 百般思念 提交于 2019-12-06 18:27:50
问题 Context : I have a simple classifier based on tf.estimator.DNNClassifier that takes text and output probabilities over an intent tags. I am able to train an export the model to a serveable as well as serve the serveable using tensorflow serving. The problem is this servable is too big (around 1GB) and so I wanted to try some tensorflow graph transforms to try to reduce the size of the files being served. Problem : I understand how to take the saved_model.pb and use freeze_model.py to create a

How do I need to modify exporting a keras model to accept b64 string to RESTful API/Google cloud ML

由 允我心安 提交于 2019-12-06 14:46:30
The complete code for exporting the model: (I've already trained it and now loading from weights file) def cnn_layers(inputs): conv_base= keras.applications.mobilenetv2.MobileNetV2(input_shape=(224,224,3), input_tensor=inputs, include_top=False, weights='imagenet') for layer in conv_base.layers[:-200]: layer.trainable = False last_layer = conv_base.output x = GlobalAveragePooling2D()(last_layer) x= keras.layers.GaussianNoise(0.3)(x) x = Dense(1024,name='fc-1')(x) x = keras.layers.BatchNormalization()(x) x = keras.layers.advanced_activations.LeakyReLU(0.3)(x) x = Dropout(0.4)(x) x = Dense(512

Hot load of models into tensorflow serving container

由 流过昼夜 提交于 2019-12-06 14:15:34
问题 I know how to load a model into a container and also I know that we can create a static config file and when we run a tensorflow serving container pass it to the container and later use one the models inside that config files but I want to know if there is any way to hot load a completely new model (not a newer version of the previous model) into a running tensorflow serving container. What I mean is we run the container with model-A and later we load model-B into the container and use it,

How to serve multiple versions of model via standard tensorflow serving docker image?

由 大憨熊 提交于 2019-12-06 13:29:08
I'm new to Tensorflow serving, I just tried Tensorflow serving via docker with this tutorial and succeeded. However, when I tried it with multiple versions, it serves only the latest version. Is it possible to do that? Or do I need to try something different? This require a ModelServerConfig, which will be supported by the next docker image tensorflow/serving release 1.11.0 (available since 5. Okt 2018). Until then, you can create your own docker image, or use tensorflow/serving:nightly or tensorflow/serving:1.11.0-rc0 as stated here . See that thread for how to implement multiple models. If

Tensorflow Serving on pretrained Keras ResNet50 model returning always same predictions

由 做~自己de王妃 提交于 2019-12-06 09:27:54
问题 I'm using the following code to export a pre-trained ResNet50 keras' model to tensorflow, for tensorflow-serving: import tensorflow as tf sess = tf.Session() from keras import backend as K K.set_session(sess) K.set_learning_phase(0) # Modelo resnet con pesos entrenados en imagenet from keras.applications.resnet50 import ResNet50 model = ResNet50(weights='imagenet') # exportar en tensorflow import os version_number = max([ int(x) for x in os.listdir('./resnet-classifier') ]) + 1 export_path =

Tensorflow Serving: Rest API returns “Malformed request” error

由 ╄→尐↘猪︶ㄣ 提交于 2019-12-06 08:07:13
问题 Tensorflow Serving server (run with docker) responds to my GET (and POST) requests with this: { "error": "Malformed request: POST /v1/models/saved_model/" } Precisely the same problem was already reported but never solved (supposedly, this is a StackOverflow kind of question, not a GitHub issue): https://github.com/tensorflow/serving/issues/1085 https://github.com/tensorflow/serving/issues/1095 Any ideas? Thank you very much. 回答1: I verified that this does not work pre-v12 and does indeed

Tensorflow predict grpc not working but RESTful API working fine

由 不羁岁月 提交于 2019-12-06 08:06:34
When i am trying to execute below piece of client code i am getting error but succeeded when calling via RESTful API end point curl -d '{"signature_name":"predict_output","instances":[2.0,9.27]}' -X POST http://10.110.110.13:8501/v1/models/firstmodel:predict Could you please correct me in below code import tensorflow as tf from tensorflow_serving.apis import predict_pb2 from tensorflow_serving.apis import prediction_service_pb2_grpc import numpy as np import grpc server = '10.110.110.13:8501' channel = grpc.insecure_channel(server) stub = prediction_service_pb2_grpc.PredictionServiceStub

How to a make a model ready for TensorFlow Serving REST interface with a base64 encoded image?

由 孤者浪人 提交于 2019-12-06 07:26:22
My understanding is that I should be able to grab a TensorFlow model from Google's AI Hub, deploy it to TensorFlow Serving and use it to make predictions by POSTing images via REST requests using curl. I could not find any bbox predictors on AI Hub at this time but I did find one on the TensorFlow model zoo: http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_coco_2018_03_29.tar.gz I have the model deployed to TensorFlow serving, but the documentation is unclear with respect to exactly what should be included in the JSON of the REST request. My understanding is that The

Tensorflow Serving Compile Error Using Docker on OSX

由 江枫思渺然 提交于 2019-12-06 05:14:59
I'm trying to install TensorFlow serving on OSX El Capitan using Docker but keep running into an error. Here is the tutorial I'm following: https://tensorflow.github.io/serving/docker.html Here is the command causing the error: bazel test tensorflow_serving/... Here's the error I'm getting: for (int i = 0; i < suffix.size(); ++i) { ^ ERROR: /root/.cache/bazel/_bazel_root/f8d1071c69ea316497c31e40fe01608c/external/tf/tensorflow/core/kernels/BUILD:212:1: C++ compilation of rule '@tf//tensorflow/core/kernels:mirror_pad_op' failed: gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE

Empty variables folder after building retrained inception SavedModel

由 时光毁灭记忆、已成空白 提交于 2019-12-05 18:50:16
I'm trying to export my retrained inception model. I've read this almost similar question here and resources mentioned there as well. But after exporting the graph, the variables folder is empty which should contains files that hold the serialized variables of the graphs (saved_model.pb is created correctly I'd say). I'm using TensorFlow 1.2.1 & Python 3.5.2. Actually I've put a simple print(tf.trainable_variables()) inside the session, but it's an empty list. Here's my function to export the graph: def export_tf_model(graph_path, export_dir): builder = tf.saved_model.builder.SavedModelBuilder
订阅 tensorflow-serving