Tensorflow, multi label accuracy calculation

Question

I am working on a multi label problem and i am trying to determine the accuracy of my model.

My model:

NUM_CLASSES         


        

Answer 1

I believe the bug in your code is in: correct_prediction = tf.equal( tf.round( pred ), tf.round( y_ ) ).

pred should be unscaled logits (i.e. without a final sigmoid).

Here you want to compare the output of sigmoid(pred) and y_ (both in the interval [0, 1]) so you have to write:

correct_prediction = tf.equal(tf.round(tf.nn.sigmoid(pred)), tf.round(y_))

---

Then to compute:

• Mean accuracy over all labels:

accuracy1 = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

• Accuracy where all labels need to be correct:

all_labels_true = tf.reduce_min(tf.cast(correct_prediction), tf.float32), 1)
accuracy2 = tf.reduce_mean(all_labels_true)

Answer 2

# to get the mean accuracy over all labels, prediction_tensor are scaled logits (i.e. with final sigmoid layer)
correct_prediction = tf.equal( tf.round( prediction_tensor ), tf.round( ground_truth_tensor ) )
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

# to get the mean accuracy where all labels need to be correct
all_labels_true = tf.reduce_min(tf.cast(correct_prediction, tf.float32), 1)
accuracy2 = tf.reduce_mean(all_labels_true)

reference: https://gist.github.com/sbrodehl/2120a95d57963a289cc23bcfb24bee1b