How to get both MSE and R2 from a sklearn GridSearchCV?

Question

I can use a GridSearchCV on a pipeline and specify scoring to either be \'MSE\' or \'R2\'. I can then access gridsearchcv._best_score

Answer 1

Added in Scikit-learn 0.19

Multi-metric scoring has been introduced in GridSearchCV. An extensive example can be found here.

When performing multi-metric scoring, you should provide 2 extra arguments:

1. A list of all the metrics that you want to use for scoring.

   For evaluating multiple metrics, either give a list of (unique) strings or a dict with names as keys and callables as values.

2. Since you can't maximize all metrics at once, you need to provide a single metric (or a custom combination of metrics) for which you want to optimize.

   For multiple metric evaluation, this needs to be a string denoting the scorer that would be used to find the best parameters for refitting the estimator at the end.

   Where there are considerations other than maximum score in choosing a best estimator, refit can be set to a function which returns the selected best_index_ given cv_results_.

In your case, you would want to use something like

cv=GridSearchCV(DecisionTreeClassifier(random_state=42),
                  param_grid={'min_samples_split': range(2, 403, 10)},
                  scoring=['neg_mean_squared_error', 'r2'], cv=5, refit='r2')
cv.fit(x,y)

You can then analyse the detailed performance with:

cv.cv_results_