Equal Error Rate in Python

Question

Could anybody tell me how could I compute Equal Error Rate(EER) from ROC Curve in python? In scikit-learn there is method to compute roc curve and auc but could not find the met

Answer 1

Copying form How to compute Equal Error Rate (EER) on ROC by Changjiang:

from scipy.optimize import brentq
from scipy.interpolate import interp1d
from sklearn.metrics import roc_curve

fpr, tpr, thresholds = roc_curve(y, y_score, pos_label=1)

eer = brentq(lambda x : 1. - x - interp1d(fpr, tpr)(x), 0., 1.)
thresh = interp1d(fpr, thresholds)(eer)

That gave me correct EER value. Also remember that in the documentation it's written that y is True binary labels in range {0, 1} or {-1, 1}. If labels are not binary, pos_label should be explicitly given and y_score is Target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions (as returned by “decision_function” on some classifiers).

Answer 2

Equal error rate (EER) is where your false pos rate (fpr) == false neg rate (fnr) [smaller is better]

using fpr, tpr and thresholds your are getting from roc sklearn computation, you can use this function to get EER:

def compute_eer(fpr,tpr,thresholds):
    """ Returns equal error rate (EER) and the corresponding threshold. """
    fnr = 1-tpr
    abs_diffs = np.abs(fpr - fnr)
    min_index = np.argmin(abs_diffs)
    eer = np.mean((fpr[min_index], fnr[min_index]))
    return eer, thresholds[min_index]

Answer 3

The EER is defined as FPR = 1 - PTR = FNR. This is wrong.

Since FPR= 1-TNR (True Negative Rate) and therefore, not equal to FNR.

Answer 4

To estimate the Equal Error Rate EER you look for the point within the ROC that makes the TPR value equal to FPR value, that is, TPR-FPR=0. In other words you look for the minimum point of abs(TPR-FPR)

1. First of all you need to estimate the ROC curve:

fpr, tpr, threshold = roc_curve(y, y_pred, pos_label=1)

2. To compute the EER in python you need only one line of code:

EER = threshold(np.argmin(abs(tpr-fpr)))

Answer 5

For any one else whom arrives here via a Google search. The Fran answer is incorrect as Gerhard points out. The correct code would be:

fpr, tpr, threshold = roc_curve(y, y_pred, pos_label=1)
fnr = 1 - tpr
eer_threshold = threshold(np.nanargmin(np.absolute((fnr - fpr))))

Note that this gets you the threshold at which the EER occurs not, the EER. The EER is defined as FPR = 1 - PTR = FNR. Thus to get the EER (the actual error rate) you could use the following:

EER = fpr(np.nanargmin(np.absolute((fnr - fpr))))

as a sanity check the value should be close to

EER = fnr(np.nanargmin(np.absolute((fnr - fpr))))

since this is an approximation.