How to design an execution engine for a sequence of tasks

Question

I am trying to code a problem in Java where I have to execute a bunch of tasks.

Problem

Execute a job which consists of multiple tasks and thos

Answer 1

2 options in terms of pattern usage, but in essence they are pretty similar. In either cases, cyclic dependencies situation need to be handled as a error in task dependency configuration. e.g. A -> B -> A

1. Mediator pattern

After each task[i] finishes, it will notify the Mediator, the the Mediator will notify all successor tasks of task[i]. The graph of task dependencies will be read when the execution engine starts as the data structure for Mediator to use.

2. Publish/Subscribe pattern via message bus.

The graph of task dependencies will be read as the engine starts, and each task will subscribe to MessageBus of its predecessor task's (task[i]) message of completion on topic[i]

When each task[i] finishes, it will send completion message to the MessageBus in topic[i]. Each task who subscribes to topic[i] will get notified and starts to work.

Answer 2

Your problem looks like a good use case for Java's ForkJoin Framework. You could implement your tasks as RecursiveActions or RecursiveTasks (depending on whether you need a return value or not) which will start their sub tasks on whatever condition you need. You'll also be able to control if your sub tasks run sequentially or in parallel.

Example:

public class TaskA extends RecursiveAction {
  // ...

  protected void compute() {
    if (conditionForTaskM) {
      TaskM m = new TaskM();
      // Run task M asynchronously or use m.invoke() to run it synchronously.
      invokeAll(m);
    }

    // Run task N at the end of A
    invokeAll(new TaskN());
  }

}

You need an instance of the ForkJoinPool to run your tasks:

public static void main(String[] args) {
  ForkJoinPool pool = new ForkJoinPool();
  pool.submit(new TaskA());

  // Properly shutdown your pool...
}

This example is quite simplistic in implementing a part of your example problem. But generally speaking, the ForkJoin Framework allows you to create tree-like structures of tasks where every parent task (such as A, B and P) allows you to control the execution of its immediate child tasks.

Answer 3

If you want to reinvent the wheel and develop the solution yourself, fine - it's your choice. However doing this properly is rather hard, especially the threading part. However, if you can consider some outside help with at least the building blocks, those can be:

• guava's ListenableFuture - using this library you can create Callables and feed them to special thread pool executor, which then allows custom callbacks on ListenableFuture completion.
• you could take a look at RX Java Observable which allows mixing and matching different tasks. This uses non-imperative coding style so beware!

Answer 4

If I understood well your need you may use a workflow engine like Activity to solve it. I think it would be easier than re-inventing a workflow engine for your specific need.