Parallel Dynamic Programming

Question

Are there any good papers discussing how to take a dynamic program and parallelize it?

Answer 1

IIRC, what you typically do with dynamic programming is to recursively divide a problem into subproblems, and assemble optimal solutions from optimal subsolutions. What makes it effective is that all optimal subsolutions are built into a cache so they need not be recomputed.

If the problem can be divided several ways, you can fork the solver for each subsolution. If each(sub) problem averages 1+epsilon (for epsilon interestingly more than zero) possible subsolutions, then you'll get a lot of parallelism this way. You'll probably need locks on the cache entries to protect the individual solutions from being constructed more than once.

You need a language in which you can fork subtasks more cheaply than the work to solve them, and which is happy to have lots of forked tasks at once. The typical parallel offerings in most languages do this badly; you can't have lots of forked tasks in systems that use "the big stack model" (see How does a stackless language work?).

We implemented our parallel programming langauge, PARLANSE, to get a language that had the right properties.