Vowpal Wabbit - How to get prediction probabilities from contextual bandit model on a test sample

Question

Given a trained contextual bandit model, how can I retrieve a prediction vector on test samples?

For example, let's say I have a train set named "train.dat" containing lines formatted as below

1:-1:0.3 | a b c  # <action:cost:probability | features> 
2:2:0.3 | a d d 
3:-1:0.3 | a b e
....

And I run below command.

vw -d train.dat --cb 30 -f cb.model --save_resume

This produces a file, 'cb.model'. Now, let's say I have a test dataset as below

| a d d 
| a b e

I'd like to see probabilities as below

0.2 0.7 0.1

The interpretation of these probabilities would be that action 1 should be picked 20% of the time, action 2 - 70%, and action 3 - 10% of the time.

Is there a way to get something like this?

Answer 1

When you use "--cb K", the prediction is the optimal arm/action based on argmax policy, which is a static policy.

When using "--cb_explore K", the prediction output contains the probability for each arm/action. Depending the policy you pick, the probabilities are calculated differently.

Answer 2

If you send those lines to a daemon running your model, you'd get just that. You send a context, and the reply is a probability distribution across the number of allowed actions, presumably comprising the "recommendation" provided by the model.

Say you have 3 actions, like in your example. Start a contextual bandits daemon:

vowpalwabbit/vw -d train.dat --cb_explore 3 -t --daemon --quiet --port 26542

Then send a context to it:

| a d d 

You'll get just what you want as the reply.