问题
Background
I have a multi-label classification problem with 5 labels (e.g. [1 0 1 1 0]
). Therefore, I want my model to improve at metrics such as fixed recall, precision-recall AUC or ROC AUC.
It doesn't make sense to use a loss function (e.g. binary_crossentropy
) that is not directly related to the performance measurement I want to optimize. Therefore, I want to use TensorFlow's global_objectives.recall_at_precision_loss()
or similar as loss function.
- Relevant GitHub: https://github.com/tensorflow/models/tree/master/research/global_objectives
- Relevant paper (Scalable Learning of Non-Decomposable Objectives): https://arxiv.org/abs/1608.04802
Not metric
I'm not looking for implementing a tf.metrics
. I already succeeded in that following: https://stackoverflow.com/a/50566908/3399066
Problem
I think my issue can be divided into 2 problems:
- How to use
global_objectives.recall_at_precision_loss()
or similar? - How to use it in a Keras model with TF backend?
Problem 1
There is a file called loss_layers_example.py
on the global objectives GitHub page (same as above). However, since I don't have much experience with TF, I don't really understand how to use it. Also, Googling for TensorFlow recall_at_precision_loss example
or TensorFlow Global objectives example
won't give me any clearer example.
How do I use global_objectives.recall_at_precision_loss()
in a simple TF example?
Problem 2
Would something like (in Keras): model.compile(loss = ??.recall_at_precision_loss, ...)
be enough?
My feeling tells me it is more complex than that, due to the use of global variables used in loss_layers_example.py
.
How to use loss functions similar to global_objectives.recall_at_precision_loss()
in Keras?
回答1:
I managed to make it work by:
- Explicitly reshaping tensors to BATCH_SIZE length (see code below)
- Cutting the dataset size to a multiple of BATCH_SIZE
def precision_recall_auc_loss(y_true, y_pred):
y_true = keras.backend.reshape(y_true, (BATCH_SIZE, 1))
y_pred = keras.backend.reshape(y_pred, (BATCH_SIZE, 1))
util.get_num_labels = lambda labels : 1
return loss_layers.precision_recall_auc_loss(y_true, y_pred)[0]
回答2:
Similar to Martino's answer, but will infer shape from input (setting it to a fixed batch size did not work for me).
The outside function isn't strictly necessary, but it feels a bit more natural to pass params as you configure the loss function, especially when your wrapper is defined in an external module.
import keras.backend as K
from global_objectives.loss_layers import precision_at_recall_loss
def get_precision_at_recall_loss(target_recall):
def precision_at_recall_loss_wrapper(y_true, y_pred):
y_true = K.reshape(y_true, (-1, 1))
y_pred = K.reshape(y_pred, (-1, 1))
return precision_at_recall_loss(y_true, y_pred, target_recall)[0]
return precision_at_recall_loss_wrapper
Then, when compiling the model:
TARGET_RECALL = 0.9
model.compile(optimizer='adam', loss=get_precision_at_recall_loss(TARGET_RECALL))
来源:https://stackoverflow.com/questions/54286334/use-tensorflow-loss-global-objectives-recall-at-precision-loss-with-keras-not