Output score , class and id Extraction using TensorFlow object detection

Question

How can I extract the output scores for objects , object class ,object id detected in images , generated by the Tensorflow Model for Object Detection ?

I want to store

Answer 1

You may need some knowledge background about tensorflow object detection, short and quick solution here might be the way you expected :

with detection_graph.as_default():
  with tf.Session(graph=detection_graph) as sess:
    for image_path in TEST_IMAGE_PATHS:
      image = Image.open(image_path)
      image_np = load_image_into_numpy_array(image)
      image_np_expanded = np.expand_dims(image_np, axis=0)
      image_tensor = detection_graph.get_tensor_by_name('image_tensor:0')
      boxes = detection_graph.get_tensor_by_name('detection_boxes:0')
      scores = detection_graph.get_tensor_by_name('detection_scores:0')
      classes = detection_graph.get_tensor_by_name('detection_classes:0')
      num_detections = detection_graph.get_tensor_by_name('num_detections:0')
      # Actual detection.
      (boxes, scores, classes, num_detections) = sess.run(
          [boxes, scores, classes, num_detections],
          feed_dict={image_tensor: image_np_expanded})
      # Visualization of the results of a detection.
      vis_util.visualize_boxes_and_labels_on_image_array(
          image_np,
          np.squeeze(boxes),
          np.squeeze(classes).astype(np.int32),
          np.squeeze(scores),
          category_index,
          use_normalized_coordinates=True,
          line_thickness=8)
      objects = []
      threshold = 0.5 # in order to get higher percentages you need to lower this number; usually at 0.01 you get 100% predicted objects
      for index, value in enumerate(classes[0]):
          object_dict = {}
          if scores[0, index] > threshold:
              object_dict[(category_index.get(value)).get('name').encode('utf8')] = \
                        scores[0, index]
              objects.append(object_dict)
      print (objects)
      print(len(np.where(scores[0] > threshold)[0])/num_detections[0])
      plt.figure(figsize=IMAGE_SIZE)
      plt.imshow(image_np)

Hope this helpful.

Answer 2

It gives you the class with the highest score because output tensors are sorted from highest score to lowest and you are asking for the highest score by indexing to the first element [0].

Look at object_detection/inference/detection_inference for inspiration.

As for class names, you can use the label map to create a category index dictionary to translate class ids to names.

Answer 3

Get class name, your label map should be able to help here.

from object_detection.utils import label_map_util

label_map_path = os.path.join(annotations_dir, 'label_map.pbtxt')
label_map_dict = label_map_util.get_label_map_dict(label_map_path)
label_map_dict_number_to_name = {v: k for k, v in label_map_dict.iteritems()}
class_number = output_dict['detection_classes'][index]
class_name = label_map_dict_number_to_name[class_number]

Please paste your code, so we can figure out why only one box is in y