Is there a way to use gradient boosting on regression using Vowpal Wabbit? I use various techniques that come with Vowpal Wabbit that are helpful. I want to try gradient boosting along with that, but I can't find a way to implement gradient boosting on VW.
The idea of gradient boosting is that an ensemble model is built from black-box weak models. You can surely use VW as the black box, but note that VW does not offer decision trees, which are the most popular choice for the black-box weak models in boosting. Boosting in general decreases bias (and increases variance), so you should make sure that the VW models have low variance (no overfitting). See bias-variance tradeoff.
There are some reductions related to boosting and bagging in VW:
--autolink Nadds a link function with polynomial N, which can be considered a simple way of boosting.--log_multi Kis an online boosting algorithm for K-class classification. See the paper. You can use it even for binary classification (K=2), but not for regression.--bootstrap MM-way bootstrap by online importance resampling. Use--bs_type=votefor classification and--bs_type=meanfor regression. Note that this is bagging, not boosting.--boosting N(added on 2015-06-17) online boosting with N weak learners, see a theoretic paper
来源：https://stackoverflow.com/questions/30008991/gradient-boosting-on-vowpal-wabbit