Java: How to get finished threads to pickup tasks from running threads

Question

I am working on a multithreaded application with tasks that have varying run times. When one thread finishes, is there a way for it to take over some tasks from a still running thread?

Here is an example. I kick off my program with 5 threads, and each have 50 tasks. When the quickest running thread finishes, another thread still has 40 tasks to complete. How can I get the finished thread to take 20 tasks from the other thread, so each continue working on 20 a piece, rather than waiting for the running thread to complete the remaining 40?

Answer 1

Use ForkJoinPool

A ForkJoinPool differs from other kinds of ExecutorService mainly by virtue of employing work-stealing: all threads in the pool attempt to find and execute subtasks created by other active tasks (eventually blocking waiting for work if none exist). This enables efficient processing when most tasks spawn other subtasks (as do most ForkJoinTasks). When setting asyncMode to true in constructors, ForkJoinPools may also be appropriate for use with event-style tasks that are never joined.

Java 8 provides one more API in Executors

static ExecutorService  newWorkStealingPool()

Creates a work-stealing thread pool using all available processors as its target parallelism level.

Have a look at this igvtia article by Ilya Grigorik for more details.

Have a look at other related java concurrent API @ tutorials like ThreadPoolExecutor, ExecutorService etc.

Answer 2

It's better to use a ThreadPoolExecutor. It will automatically assign tasks to free Threads.

Answer 3

Use thread pools, which are created thanks to the Executors class:

 ExecutorService es = Executors.newFixedThreadPool(5);
 List<Runnable> tasks = // create your 50 runnable
 List<Future<?>> futures = new ArrayList<>(tasks.size());
 for(Runnable r : tasks) {
     Future<?> f = es.submit(t);
     futures.add(f);
 }

The documentation explains quite well how it works so I recommend you give it a look.

Answer 4

Do not allow threads to take on more than one task. In this way, any thread that completes its task takes the next available task in the queue. These threads are not being created new for each task, but instead are being re-used, so there is very little overhead.

Consider - 2 threads have 20 tasks each, and you want the second thread to take over tasks from the first if it is not yet finished. Compare this to having 40 tasks in a queue being serviced by 2 threads, meaning tasks are always going to be executed as soon as possible without the complexity of trying to move them between threads.

I'm not seeing the logic in the question - if you have groups of tasks that cannot be multi-threaded due to sequencing issues (which is the only reason I can see to submit a task that is itself a group of tasks onto the queue), then you can not have another thread take over unfinished processing (because then the sequencing of the overall group will be broken). If you don't require sequenced processing, then throw all the tasks onto the queue and let them be executed as soon as possible.

If you always want the tasks from a given group to execute sooner, assign them a higher priority and use a Priority Queue serviced by multiple threads.