问题
What's the proper way to nest calls to GNU parallel?
Silly example:
seq 1 100 | parallel echo {} | parallel seq {} 1000
My understanding is that on an 8-CPU box, each parallel would launch 8 jobs for a total of 64 jobs. If you're calling something more substantial than seq this could potentially overload the box. Is there a way to limit the number of jobs but still make full use of parallelism?
回答1:
Use -j to limit either the outer or the inner parallel:
seq 1 100 | parallel -j1 "echo {} | parallel -I // seq // 1000"
Often you can express the nested command using mulitple :::: or ::: like this:
parallel seq {1} {3} {2} :::: <(seq 10) <(seq 20 30) ::: 1 2 3
It is better because you will keep 8 jobs running at all time where as the above will in periods run fewer than 8 jobs on an 8 core machine.
来源：https://stackoverflow.com/questions/23411266/calling-gnu-parallel-from-gnu-parallel