How can I use the parallel command to exploit multi-core parallelism on my MacBook?

Question

I often use the find command on Linux and macOS. I just discovered the command parallel, and I would like to combine it with find command

Answer 1

As others have written find is I/O heavy and most likely not limited by your CPUs.

But depending on your disks it can be better to run the jobs in parallel.

NVMe disks are known for performing best if there are 4-8 accesses running in parallel. Some network file systems also work faster with multiple processes.

So some level of parallelization can make sense, but you really have to measure to be sure.

To parallelize find with 8 jobs running in parallel:

parallel -j8  find {} ::: *

This works best if you are in a dir that has many subdirs: Each subdir will then be searched in parallel. Otherwise this may work better:

parallel -j8  find {} ::: */*

Basically the same idea, but now using subdirs of dirs.

If you want the results printed as soon as they are found (and not after the find is finished) use --line-buffer (or --lb):

parallel --lb -j8  find {} ::: */*

To learn about GNU Parallel spend 20 minutes reading chapter 1+2 of https://doi.org/10.5281/zenodo.1146014 and print the cheat sheet: https://www.gnu.org/software/parallel/parallel_cheat.pdf

Your command line will thank you for it.