问题
I'm trying to do a feature selection by chi-square method in scikit-learn (sklearn.feature_selection.SelectKBest). When I'm trying to apply this to a multilabel problem, I get this warning:
UserWarning: Duplicate scores. Result may depend on feature ordering.There are probably duplicate features, or you used a classification score for a regression task.
warn("Duplicate scores. Result may depend on feature ordering."
Why is it appearning and how to properly apply feature selection is this case?
回答1:
The code warns you that arbitrary tie-breaking may need to be performed because some features have exactly the same score.
That said, feature selection does not actually work for multilabel out of the box; the best you can currently do is tie feature selection and a classifier together in a pipeline, then feed that to a multilabel meta-estimator. Example (untested):
clf = Pipeline([('chi2', SelectKBest(chi2, k=1000)),
('svm', LinearSVC())])
multi_clf = OneVsRestClassifier(clf)
(This warning is, I believe, issued even when the tied features aren't actually the k'th and (k+1)'th, I think. It can usually be ignored safely.)
回答2:
I know the topic is a bit old but the following is working for me:
clf = Pipeline([('chi2', SelectKBest(chi2, k=1000)),
('lasso', OneVsRestClassifier(LogisticRegression()))])
来源:https://stackoverflow.com/questions/16400722/feature-selection-for-multilabel-classification-scikit-learn