vowpalwabbit

Is there a way to use gradient boosting on regression using Vowpal Wabbit? I use various techniques that come with Vowpal Wabbit that are helpful. I want to try gradient boosting along with that, but I can't find a way to implement gradient boosting on VW. The idea of gradient boosting is that an ensemble model is built from black-box weak models. You can surely use VW as the black box, but note that VW does not offer decision trees, which are the most popular choice for the black-box weak models in boosting. Boosting in general decreases bias (and increases variance), so you should make sure

I have started using Vowpal Wabbit for logistic regression, however I am unable to reproduce the results it gives. Perhaps there is some undocumented "magic" it does, but has anyone been able to replicate / verify / check the calculations for logistic regression? For example, with the simple data below, we aim to model the way age predicts label . It is obvious there is a strong relationship as when age increases the probability of observing 1 increases. As a simple unit test, I used the 12 rows of data below: age label 20 0 25 0 30 0 35 0 40 0 50 0 60 1 65 0 70 1 75 1 77 1 80 1 Now,

I have the following data with all categorical variables: class education income social_standing 1 basic low good 0 low high V_good 1 high low not_good 0 v_high high good Here education has four levels (basic, low, high and v_high). income has two levels low and high ; and social_standing has three levels (good, v_good and not_good). In so far as my understanding of converting the above data to VW format is concerned, it will be something like this: 1 |person education_basic income_low social_standing_good 0 |person education_low income_high social_standing_v_good 1 |person education_high

I am trying to use vowpal wabbit for logistic regression. I am not sure if this is the right syntax to do it For training, I do ./vw -d ~/Desktop/new_data.txt --passes 20 --binary --cache_file cache.txt -f lr.vw --loss_function logistic --l1 0.05 For testing I do ./vw -d ~/libsvm-3.18_test/matlab/new_data_test.txt --binary -t -i lr.vw -p predictions.txt -r raw_score.txt Here is a snippet from my train data -1:1.00038 | 110:0.30103 262:0.90309 689:1.20412 1103:0.477121 1286:1.5563 2663:0.30103 2667:0.30103 2715:4.63112 3012:0.30103 3113:8.38411 3119:4.62325 3382:1.07918 3666:1.20412 3728:5