Using sklearn voting ensemble with partial fit

Question

Can someone please tell how to use ensembles in sklearn using partial fit. I don\'t want to retrain my model. Alternatively, can we pass pre-trained models for ensembling ?

Answer 1

Unfortunately, currently this is not possible in scikit VotingClassifier.

But you can use http://sebastianraschka.com/Articles/2014_ensemble_classifier.html (from which VotingClassifer is implemented) to try and implement your own voting classifier which can take pre-fitted models.

Also we can look at the source code here and modify it to our use:

from sklearn.preprocessing import LabelEncoder
import numpy as np

le_ = LabelEncoder()

# When you do partial_fit, the first fit of any classifier requires 
all available labels (output classes), 
you should supply all same labels here in y.
le_.fit(y)

# Fill below list with fitted or partial fitted estimators
clf_list = [clf1, clf2, clf3, ... ]

# Fill weights -> array-like, shape = [n_classifiers] or None
weights = [clf1_wgt, clf2_wgt, ... ]
weights = None

#For hard voting:
pred = np.asarray([clf.predict(X) for clf in clf_list]).T
pred = np.apply_along_axis(lambda x:
                           np.argmax(np.bincount(x, weights=weights)),
                           axis=1,
                           arr=pred.astype('int'))

#For soft voting:
pred = np.asarray([clf.predict_proba(X) for clf in clf_list])
pred = np.average(pred, axis=0, weights=weights)
pred = np.argmax(pred, axis=1)

#Finally, reverse transform the labels for correct output:
pred = le_.inverse_transform(np.argmax(pred, axis=1))