Multiple parameter optimization with lots of local minima

Question

I\'m looking for algorithms to find a \"best\" set of parameter values. The function in question has a lot of local minima and changes very quickly. To make matters even worse,

Answer 1

Your situation seems to be similar to that of the poster of Software to Tune/Calibrate Properties for Heuristic Algorithms, and I would give you the same advice I gave there: consider a Metropolis-Hastings like approach with multiple walkers and a simulated annealing of the step sizes.

The difficulty in using a Monte Carlo methods in your case is the expensive evaluation of each candidate. How expensive, compared to the time you have at hand? If you need a good answer in a few minutes this isn't going to be fast enough. If you can leave it running over night, it'll work reasonably well.

Given a complicated search space, I'd recommend a random initial distributed. You final answer may simply be the best individual result recorded during the whole run, or the mean position of the walker with the best result.

Don't be put off that I was discussing maximizing there and you want to minimize: the figure of merit can be negated or inverted.