How to use ModelCheckpoint with custom metrics in Keras?

Question

Is it possible to use custom metrics in the ModelCheckpoint callback?

Answer 1

Yes, it is possible.

Define the custom metrics as described in the documentation:

import keras.backend as K

def mean_pred(y_true, y_pred):
    return K.mean(y_pred)

model.compile(optimizer='rmsprop',
              loss='binary_crossentropy',
              metrics=['accuracy', mean_pred])

To check all available metrics:

print(model.metrics_names)
> ['loss', 'acc', 'mean_pred']

Pass the metric name to ModelCheckpoint through monitor. If you want the metric calculated in the validation, use the val_ prefix.

ModelCheckpoint(weights.{epoch:02d}-{val_mean_pred:.2f}.hdf5,
                monitor='val_mean_pred',
                save_best_only=True,
                save_weights_only=True,
                mode='max',
                period=1)

Don't use mode='auto' for custom metrics. Understand why here.

---

Why am I answering my own question? Check this.