What's the difference between predict_proba and decision_function in scikit-learn?

Question

I\'m studying a scikit-learn example (Classifier comparison) and got confused with predict_proba and decision_function.

They plot the classific

Answer 1

Your example is

if hasattr(clf, "decision_function"):
    Z = clf.decision_function(np.c_[xx.ravel(), yy.ravel()])
else:
    Z = clf.predict_proba(np.c_[xx.ravel(), yy.ravel()])[:, 1]

so the code uses decision_function if it exists. On the SVM case, predict_proba is computed (in the binary case)

using Platt scaling

which is both "expensive" and has "theoretical issues". That's why decision_function is used here. (as @Ami said, this is the margin - the distance to the hyperplane, which is accessible without much further computation). In the SVM case, it is advised to

use decision_function instead of predict_proba.

There are other decision_functions: SGDClassifier's. Here, predict_proba depends on the loss function, and decision_function is universally available.