How to design an execution engine for a sequence of tasks

Question

I am trying to code a problem in Java where I have to execute a bunch of tasks.

Problem

Execute a job which consists of multiple tasks and thos

Answer 1

There is a framework specifically dedicated for this purpose, called Dexecutor, with Dexecutor, you model your requirements in terms of graph, when in comes to execution, dexecutor would take care of it executing it in reliable way.

For example :

@Test
public void testDependentTaskExecution() {

    ExecutorService executorService = newExecutor();
    ExecutionEngine executionEngine = new DefaultExecutionEngine<>(executorService);

    try {
        DefaultDependentTasksExecutor executor = new DefaultDependentTasksExecutor(
                executionEngine, new SleepyTaskProvider());

        executor.addDependency(1, 2);
        executor.addDependency(1, 2);
        executor.addDependency(1, 3);
        executor.addDependency(3, 4);
        executor.addDependency(3, 5);
        executor.addDependency(3, 6);
        executor.addDependency(2, 7);
        executor.addDependency(2, 9);
        executor.addDependency(2, 8);
        executor.addDependency(9, 10);
        executor.addDependency(12, 13);
        executor.addDependency(13, 4);
        executor.addDependency(13, 14);
        executor.addIndependent(11);

        executor.execute(ExecutionConfig.NON_TERMINATING);

        Collection> processedNodesOrder = Deencapsulation.getField(executor, "processedNodes");
        assertThat(processedNodesOrder).containsAll(executionOrderExpectedResult());
        assertThat(processedNodesOrder).size().isEqualTo(14);

    } finally {
        try {
            executorService.shutdownNow();
            executorService.awaitTermination(1, TimeUnit.SECONDS);
        } catch (InterruptedException e) {

        }
    }
}

private Collection> executionOrderExpectedResult() {
    List> result = new ArrayList>();
    result.add(new Node(1));
    result.add(new Node(2));
    result.add(new Node(7));
    result.add(new Node(9));
    result.add(new Node(10));
    result.add(new Node(8));
    result.add(new Node(11));
    result.add(new Node(12));
    result.add(new Node(3));
    result.add(new Node(13));
    result.add(new Node(5));
    result.add(new Node(6));
    result.add(new Node(4));
    result.add(new Node(14));
    return result;
}

private ExecutorService newExecutor() {
    return Executors.newFixedThreadPool(ThreadPoolUtil.ioIntesivePoolSize());
}

private static class SleepyTaskProvider implements TaskProvider {

    public Task provideTask(final Integer id) {

        return new Task() {

            private static final long serialVersionUID = 1L;

            public Integer execute() {
                if (id == 2) {
                    throw new IllegalArgumentException("Invalid task");
                }
                return id;
            }
        };
    }
}

Here is the modeled graph

This means tasks # 1 , 12 and 11 would run in parallel, once one of them finishes its dependent tasks would kick off, for example once task#1 finishes, its dependent tasks #2 and #3 would kick off