How to retrieve float_val from a PredictResponse object?

Question

I am running a prediction on a tensorflow-serving model, and I get back this PredictResponse object as output:

Result:

outputs {
  key: "outputs"
  value {
    dtype: DT_FLOAT
    tensor_shape {
      dim {
        size: 1
      }
      dim {
        size: 20
      }
    }
    float_val: 0.000343723397236
    float_val: 0.999655127525
    float_val: 3.96821117632e-11
    float_val: 1.20521548297e-09
    float_val: 2.09611101809e-08
    float_val: 1.46216549979e-09
    float_val: 3.87274603497e-08
    float_val: 1.83520256769e-08
    float_val: 1.47733780764e-08
    float_val: 8.00914179422e-08
    float_val: 2.29388191997e-07
    float_val: 6.27798826258e-08
    float_val: 1.08802950649e-07
    float_val: 4.39628813353e-08
    float_val: 7.87182985462e-10
    float_val: 1.31638898893e-07
    float_val: 1.42612295306e-08
    float_val: 3.0768305237e-07
    float_val: 1.12661648899e-08
    float_val: 1.68554503688e-08
  }
}

I would like to get the out the float vals as a list. Or, alternatively, return the value/index of the argmax float_val!

This is generated by:

stub = prediction_service_pb2.beta_create_PredictionService_stub(channel) result = stub.Predict(request, 200.0)

Thanks for your help in advance.

Answer 1

The answer is:

floats = result.outputs['outputs'].float_val

Answer 2

If you would like to convert the entire PredictResponse to a numpy array (including its dimentions)

<script src="https://gist.github.com/eavidan/22ad044f909e5739ceca9ff9e6feaa43.js"></script>

Answer 3

This answer is for tensorflow-serving-api-python3 1.8.0

result.outputs['your key name'].float_val #key name in your case is outputs

This will return a repeated scalar container object. Which can be passed to python list() or np.array() etc

Answer 4

result["outputs"].float_val should be a python list

Answer 5

You generally want to recover a tensor, with a shape (not just a long list of floats). Here's how:

outputs_tensor_proto = result.outputs["outputs"]
shape = tf.TensorShape(outputs_tensor_proto.tensor_shape)
outputs = tf.constant(outputs_tensor_proto.float_val, shape=shape)

If you prefer to get a NumPy array, then just replace the last line:

outputs = np.array(outputs_tensor_proto.float_val).reshape(shape.as_list())

If you don't want to depend on the TensorFlow library at all, for some reason:

outputs_tensor_proto = result.outputs["outputs"]
shape = [dim.size for dim in outputs_tensor_proto.tensor_shape.dim]
outputs = np.array(outputs_tensor_proto.float_val).reshape(shape)