PyMC2 and PyMC3 give different results…?

问题

I'm trying to get a simple PyMC2 model working in PyMC3. I've gotten the model to run but the models give very different MAP estimates for the variables. Here is my PyMC2 model:

import pymc
theta = pymc.Normal('theta', 0, .88)

X1 = pymc.Bernoulli('X2', p=pymc.Lambda('a', lambda theta=theta:1./(1+np.exp(-(theta-(-0.75))))), value=[1],observed=True)
X2 = pymc.Bernoulli('X3', p=pymc.Lambda('b', lambda theta=theta:1./(1+np.exp(-(theta-0)))), value=[1],observed=True)

model = pymc.Model([theta, X1, X2])
mcmc = pymc.MCMC(model)
mcmc.sample(iter=25000, burn=5000)
trace = (mcmc.trace('theta')[:])
print "\nThe MAP value for theta is", trace.sum()/len(trace)

That seems to work as expected. I had all sorts of trouble figuring out how to use the equivalent of the pymc.Lambda object in PyMC3. I eventually came across the Deterministic object. The following is my code:

import pymc3

with pymc3.Model() as model:

    theta = pymc3.Normal('theta', 0, 0.88)
    X1 = pymc3.Bernoulli('X1', p=pymc3.Deterministic('b', 1./(1+np.exp(-(theta-(-0.75))))), observed=[1])
    X2 = pymc3.Bernoulli('X2', p=pymc3.Deterministic('c', 1./(1+np.exp(-(theta-(0))))), observed=[1])

    start=pymc3.find_MAP()
    step=pymc3.NUTS(state=start)
    trace = pymc3.sample(20000, step, njobs=1, progressbar=True)

pymc3.traceplot(trace)

The problem I'm having is that my MAP estimate for theta using PyMC2 is ~0.68 (correct), while the estimate PyMC3 gives is ~0.26 (incorrect). I suspect this has something to do with the way I'm defining the deterministic function. PyMC3 won't let me use a lambda function, so I just have to write the expression in-line. When I try to use lambda theta=theta:... I get this error:

AsTensorError: ('Cannot convert <function <lambda> at 0x157323e60> to TensorType', <type 'function'>)

Something to do with Theano?? Any suggestions would be greatly appreciated!

回答1:

It works when you use a theano tensor instead of a numpy function in your Deterministic.

import pymc3
import theano.tensor as tt

with pymc3.Model() as model:

    theta = pymc3.Normal('theta', 0, 0.88)
    X1 = pymc3.Bernoulli('X1', p=pymc3.Deterministic('b', 1./(1+tt.exp(-(theta-(-0.75))))), observed=[1])
    X2 = pymc3.Bernoulli('X2', p=pymc3.Deterministic('c', 1./(1+tt.exp(-(theta-(0))))), observed=[1])

    start=pymc3.find_MAP()
    step=pymc3.NUTS(state=start)
    trace = pymc3.sample(20000, step, njobs=1, progressbar=True)

print "\nThe MAP value for theta is", np.median(trace['theta'])

pymc3.traceplot(trace);

Here's the output:

回答2:

Just in case someone else has the same problem, I think I found an answer. After trying different sampling algorithms I found that:

find_MAP gave the incorrect answer
the NUTS sampler gave the incorrect answer
the Metropolis sampler gave the correct answer, yay!

I read somewhere else that the NUTS sampler doesn't work with Deterministic. I don't know why. Maybe that's the case with find_MAP too? But for now I'll stick with Metropolis.

回答3:

Also, NUTS doesn't handle discrete variables. If you want to use NUTS, you have to split up the samplers:

step1 = pymc3.NUTS([theta])
step2 = pymc3.BinaryMetropolis([X1,X2])

trace = pymc3.sample(10000, [step1, step2], start)

EDIT: Missed that 'b' and 'c' were defined inline. Removed them from the NUTS function call

回答4:

The MAP value is not defined as the mean of a distribution, but as its maximum. With pymc2 you can find it with:

M = pymc.MAP(model)
M.fit()
theta.value

which returns array(0.6253614422469552)

This agrees with the MAP that you find with find_MAP in pymc3, which you call start:

{'theta': array(0.6253614811102668)}

The issue of which is a better sampler is a different one, and does not depend on the calculation of the MAP. The MAP calculation is an optimization. See: https://pymc-devs.github.io/pymc/modelfitting.html#maximum-a-posteriori-estimates for pymc2.

来源：https://stackoverflow.com/questions/32304160/pymc2-and-pymc3-give-different-results

标签

python

pymc

pymc3