tensorflow-serving

I am trying to use Google Cloud ML to host a Tensorflow model and get predictions. I have a pretrained model that I have uploaded to the cloud and I have created a model and version in my Cloud ML console. I followed the instructions from here to prepare my data for requesting online predictions. For both the Python method and the glcoud method I get the same error. For simplicity, I'll post the gcloud method: I run gcloud ml-engine predict --model spell_correction --json-instances test.json where test.json is my input data file (a JSON array named instances ). I get the following result:

I am trying to convert a keras model to use it for predictions on google cloud's ml-engine. I have a pre-trained classifier that takes in a numpy array as input. The normal working data I send to model.predict is named input_data. I convert it to base 64 and dump it to a json file using the following few lines: data = {} data['image_bytes'] = [{'b64':base64.b64encode(input_data.tostring())}] with open('weights/keras/example.json', 'w') as outfile: json.dump(data, outfile) Now, I try to create the TF model from my existing model: from keras.models import model_from_json import tensorflow as tf

I have a Keras model I converting to a tensorflow serving model. I can successfully convert my pretrained keras model to take b64 input, preprocess that input, and feed it to my model. My problem is that I don't know how to take the prediction data I am getting (which is enormous) and only export the top result. I am doing image segmentation so my output prediction is of shape (?, 473, 473, 3) and I'd like to get the top result and return it in b64 encoded format. What I have currently that just returns the entire prediction: sess = K.get_session() g = sess.graph g_def = graph_util.convert

So in TensorFlow's guide for using GPUs there is a part about using multiple GPUs in a "multi-tower fashion": ... for d in ['/device:GPU:2', '/device:GPU:3']: with tf.device(d): # <---- manual device placement ... Seeing this, one might be tempted to leverage this style for multiple GPU training in a custom Estimator to indicate to the model that it can be distributed across multiple GPUs efficiently. To my knowledge, if manual device placement is absent TensorFlow does not have some form of optimal device mapping (expect perhaps if you have the GPU version installed and a GPU is available,

I am running a prediction on a tensorflow-serving model, and I get back this PredictResponse object as output: Result: outputs { key: "outputs" value { dtype: DT_FLOAT tensor_shape { dim { size: 1 } dim { size: 20 } } float_val: 0.000343723397236 float_val: 0.999655127525 float_val: 3.96821117632e-11 float_val: 1.20521548297e-09 float_val: 2.09611101809e-08 float_val: 1.46216549979e-09 float_val: 3.87274603497e-08 float_val: 1.83520256769e-08 float_val: 1.47733780764e-08 float_val: 8.00914179422e-08 float_val: 2.29388191997e-07 float_val: 6.27798826258e-08 float_val: 1.08802950649e-07 float

As the tensorflow paper states, Tensorflow' cross-device communication is achieved by adding "receive node" and "send node" into devices. From my understanding, the device(Please considering only CPU devices are involved) is responsible for performing the computation of an operation. However,the data(ex:Tensor produced from an operation, Variable buffer) resides in memory. I don't know how data transfer from one device to another device is achieved physically . I guess the data transfer is achieved by shared memory. Is that right? I will appreciate any explanation/corresponding codes regarding

I have exported a SavedModel and now I with to load it back in and make a prediction. It was trained with the following features and labels: F1 : FLOAT32 F2 : FLOAT32 F3 : FLOAT32 L1 : FLOAT32 So say I want to feed in the values 20.9, 1.8, 0.9 get a single FLOAT32 prediction. How do I accomplish this? I have managed to successfully load the model, but I am not sure how to access it to make the prediction call. with tf.Session(graph=tf.Graph()) as sess: tf.saved_model.loader.load( sess, [tf.saved_model.tag_constants.SERVING], "/job/export/Servo/1503723455" ) # How can I predict from here? # I