how to fit a method belonging to an instance with pymc3?

Question

I failed to fit a method belonging to an instance of a class, as a Deterministic function, with PyMc3. Can you show me how to do that ?

For simplicity, my case is su

Answer 1

theano.compile.ops.as_op is just a short-hand for defining simple Theano Ops. If you want to code more involved ones, it is better to define it in a separate class. Objects of this class could of course take a reference to an instance of your cprofile if that really is necessary.

http://deeplearning.net/software/theano/extending/extending_theano.html

Answer 2

I finally converged toward the successful code below:

import numpy as np
import theano
from scipy.interpolate import interp1d
import pymc3 as pm3
theano.config.compute_test_value = 'ignore'
theano.config.on_unused_input = 'ignore'

class cprofile:
    observations = np.array([6.25,2.75,1.25,1.25,1.5,1.75,1.5,1])
    x = np.arange(0,18,0.5)
    observed_x = np.array([0.3,1.4,3.1,5,6.8,9,13.4,17.1])    

    def doMAP(self):
        model = pm3.Model()
        with model:
            t = pm3.Uniform("t",0,5)
            y = pm3.Uniform("y",0,5)
            z = pm3.Uniform("z",0,5)
            obs=pm3.Normal('obs',
              mu=FunctionIWantToFit(self)(t,y,z),
              sd=0.1,observed=self.observations)
            start = pm3.find_MAP()
            print('start: ',start)

class FunctionIWantToFit(theano.gof.Op):
    itypes=[theano.tensor.dscalar,
            theano.tensor.dscalar,
            theano.tensor.dscalar]
    otypes=[theano.tensor.dvector]

    def __init__(self, cp):
        self.cp = cp # note cp is an instance of the 'cprofile' class

    def perform(self,node, inputs, outputs):
        t, y, z = inputs[0], inputs[1], inputs[2]

        xxx = self.cp.x
        temp = t+y*xxx+z*xxx**2
        interpolated_concentration = interp1d(xxx,temp)   
        outputs[0][0] = interpolated_concentration(self.cp.observed_x)

testcp=cprofile()
testcp.doMAP()

thanks to the answer by Dario because I was too slow to understand the first answer by myself. I get it retrospectively but I strongly think the pymc3 doc is painfully unclear. It should contain very simple and illustrative examples.

However I didn’t succed in doing anything that work following the comment by Chris. Could anyone explain and/or give an example ?

One more thing: I don’t know whether my example above is efficient or could be simplified. In particular it gives me the impression the instance ‘testcp’ is copied twice in memory. More comments/answers are welcome to go further.

Answer 3

Ok, let's do this by parts. First I'll explain the error messages that you got, and then I'll tell you how I would proceed.

On the first question, the direct reason why you're getting a complaint on the missing parameters is because your function, defined inside the class, takes as input (self, t, y, z), while you're declaring it in the op decorator as having only three inputs (t, y, z). You would have to declare the inputs as being four in your decorator to account for the class instance itself.

On "added on april 9, 2017:", the first code will not work because the output of test.testfunc(t,y,z) is a theano function itself. pymc3.Deterministic is expecting it to output theano variables (or python variables). Instead, make test.testfun output val = t2 + y2 * self.observed_x + z2 * self.observed_x**2 directly.

Then, on "if I change 'testfunc' into:", you get that error because of the way pymc3 is trying to work with theano functions. Long story short, the problem is that when pymc3 is making use of this function, it will send it theano variables, while fval is expecting numerical variables (numpy arrays or other). As in the previous paragraph, you just need to output val directly: no need to compile any theano function for this.

As for how I would proceed, I would try to declare the class instance as input to the theano decorator. Unfortunately, I can't find any documentation on how to do this and it might actually be impossible (see this old post, for example).

Then I would try to pass everything the function needs as inputs and define it outside of the class. This could be quite cumbersome and if it needs methods as input, then you run into additional problems.

Another way of doing this is to create a child class of theano.gof.Op whose init method takes your class (or rather an instance of it) as input and then define your perform() method. This would look something like this:

class myOp(theano.gof.Op):
    """ These are the inputs/outputs you used in your as_op
    decorator.
    """
    itypes=[tt.dscalar,tt.dscalar,tt.dscalar]
    otypes=[tt.dvector]
    def __init__(self, myclass):
        """ myclass would be the class you had from before, which
        you called cprofile in your first block of code."""
        self.myclass = myclass
    def perform(self,node, inputs, outputs):
        """ Here you define your operations, but instead of
        calling everyting from that class with self.methods(), you
        just do self.myclass.methods().

        Here, 'inputs' is a list with the three inputs you declared
        so you need to unpack them. 'outputs' is something similar, so
        the function doesn't actually return anything, but saves all
        to outputs. 'node' is magic juice that keeps the world
        spinning around; you need not do anything with it, but always
        include it.
        """
        t, y, z = inputs[0][0], inputs[0][1], inputs[0][2]
        outputs[0][0] = t+y*self.myclass.x+z*self.myclass.x**2
myop = myOp(myclass)

Once you have done this, you can use myop as your Op for the rest of your code. Note that some parts are missing. You can check my example for more details.

As for the example, you do not need to define the grad() method. Because of this, you can do all operations in perform() in pure python, if that helps.

Alternatively, and I say this with a smirk on my face, if you have access to the definition of the class you're using, you can also make it inherit from theano.gof.Op, create the perform() method (as in my other example, where you left a message) and try to use it like that. It could create conflicts with whatever else you're doing with that class and it's probably quite hard to get right, but might be fun to try.