print currently evaluated params during scipy minimization

Question

I am trying to minimize a function in Python using scipy.optimize.minimize, to determine four distinct parameters.

I would like to print the currently evaluated params at each step of the optimisation algorithm, so I can use them to make my initial guess better.

How can I do this?

Answer 1

Use the callback keyword argument.

scipy.optimize.minimize can take a keyword argument callback. This should be a function that accepts, as input, the current vector of parameters. This function is called after every iteration.

For instance,

from scipy.optimize import minimize

def objective_function(xs):
    """ Function to optimize. """
    x, y = xs
    return (x-1)**2 + (y-2)**4

def print_callback(xs):
    """
    Callback called after every iteration.

    xs is the estimated location of the optimum.
    """
    print xs

minimize(objective_function, x0 = (0., 0.), callback=print_callback)

Often, one wants to retain information between different calls to the callback, such as, for instance, the iteration number. One way to do this is to use a closure:

def generate_print_callback():
    """
    Generate a callback that prints 

        iteration number | parameter values | objective function

    every tenth iteration.
    """
    saved_params = { "iteration_number" : 0 }
    def print_callback(xs):
        if saved_params["iteration_number"] % 10 == 0:
            print "{:3} | {} | {}".format(
                saved_params["iteration_number"], xs, objective_function(xs))
        saved_params["iteration_number"] += 1
    return print_callback

Call the minimize function with:

minimize(objective_function, x0 = (0., 0.), callback=generate_print_callback())