make: Run several tasks in parallel and wait for completion

Question

One target in my makefile is a very CPU and time consuming task. But I can split the workload and run the task several times in parallel to speed up the entire process.

Answer 1

Each individual logical line in a recipe is invoked in its own shell. So your makefile is basically running this:

test:
        /bin/sh -c 'echo "START"'
        /bin/sh -c './myTask.sh 5 &'
        /bin/sh -c './myTask.sh 15 &'
        /bin/sh -c './myTask.sh 10 &'
        /bin/sh -c 'wait'
        /bin/sh -c 'echo "DONE"'

You should make it all run in a single shell using semicolons and backslash/newlines to combine the physical lines into one logical line:

test:
        echo "START"; \
        ./myTask.sh 5 & \
        ./myTask.sh 15 & \
        ./myTask.sh 10 & \
        wait; \
        echo "DONE"

Answer 2

Why not use the mechanisms already built in make? Something like this:

task%:
        sh task.sh $*

test: task5 task15 task10
        echo "DONE"

You will be able to adjust the level of parallelism on the make command line, instead of having it hard-coded into your makefile (e.g use 'make -j 2 test' if you have 2 cores available and 'make -j 32 test' if you have 32 cores)