Use TensorFlow loss Global Objectives (recall_at_precision_loss) with Keras (not metrics)

☆樱花仙子☆ 提交于 2020-07-21 03:31:05

问题


Background

I have a multi-label classification problem with 5 labels (e.g. [1 0 1 1 0]). Therefore, I want my model to improve at metrics such as fixed recall, precision-recall AUC or ROC AUC.

It doesn't make sense to use a loss function (e.g. binary_crossentropy) that is not directly related to the performance measurement I want to optimize. Therefore, I want to use TensorFlow's global_objectives.recall_at_precision_loss() or similar as loss function.

  • Relevant GitHub: https://github.com/tensorflow/models/tree/master/research/global_objectives
  • Relevant paper (Scalable Learning of Non-Decomposable Objectives): https://arxiv.org/abs/1608.04802

Not metric

I'm not looking for implementing a tf.metrics. I already succeeded in that following: https://stackoverflow.com/a/50566908/3399066

Problem

I think my issue can be divided into 2 problems:

  1. How to use global_objectives.recall_at_precision_loss() or similar?
  2. How to use it in a Keras model with TF backend?

Problem 1

There is a file called loss_layers_example.py on the global objectives GitHub page (same as above). However, since I don't have much experience with TF, I don't really understand how to use it. Also, Googling for TensorFlow recall_at_precision_loss example or TensorFlow Global objectives example won't give me any clearer example.

How do I use global_objectives.recall_at_precision_loss() in a simple TF example?

Problem 2

Would something like (in Keras): model.compile(loss = ??.recall_at_precision_loss, ...) be enough? My feeling tells me it is more complex than that, due to the use of global variables used in loss_layers_example.py.

How to use loss functions similar to global_objectives.recall_at_precision_loss() in Keras?


回答1:


I managed to make it work by:

  • Explicitly reshaping tensors to BATCH_SIZE length (see code below)
  • Cutting the dataset size to a multiple of BATCH_SIZE
    def precision_recall_auc_loss(y_true, y_pred):
        y_true = keras.backend.reshape(y_true, (BATCH_SIZE, 1)) 
        y_pred = keras.backend.reshape(y_pred, (BATCH_SIZE, 1))   
        util.get_num_labels = lambda labels : 1
        return loss_layers.precision_recall_auc_loss(y_true, y_pred)[0]



回答2:


Similar to Martino's answer, but will infer shape from input (setting it to a fixed batch size did not work for me).

The outside function isn't strictly necessary, but it feels a bit more natural to pass params as you configure the loss function, especially when your wrapper is defined in an external module.

import keras.backend as K
from global_objectives.loss_layers import precision_at_recall_loss

def get_precision_at_recall_loss(target_recall): 
    def precision_at_recall_loss_wrapper(y_true, y_pred):
        y_true = K.reshape(y_true, (-1, 1)) 
        y_pred = K.reshape(y_pred, (-1, 1))   
        return precision_at_recall_loss(y_true, y_pred, target_recall)[0]
    return precision_at_recall_loss_wrapper

Then, when compiling the model:

TARGET_RECALL = 0.9
model.compile(optimizer='adam', loss=get_precision_at_recall_loss(TARGET_RECALL))


来源:https://stackoverflow.com/questions/54286334/use-tensorflow-loss-global-objectives-recall-at-precision-loss-with-keras-not

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!