Automatically setting jobs (-j) flag for a multicore machine?

Question

I have a Makefile on a machine that has a ton of cores in it, but I always seem to forget to write -jX when compiling my project and it takes way longer than it

Answer 1

As Jeremiah Willcock said, use MAKEFLAGS, but here is how to do it:

export MAKEFLAGS="-j $(grep -c ^processor /proc/cpuinfo)"

or you could just set a fixed value like this:

export MAKEFLAGS="-j 8"

If you really want to boost performance you should use ccache by adding something like the following to your Makefile:

CCACHE_EXISTS := $(shell ccache -V)
ifdef CCACHE_EXISTS
    CC := ccache $(CC)
    CXX := ccache $(CXX)
endif

Answer 2

I usually do this as follows in my bash scripts:

make -j$(nproc)

Answer 3

Aliases are not expanded inside scripts. It's better to create a separate make script and place it into one of the $PATH directories:

#!/bin/sh

if [ -f /proc/cpuinfo ]; then
    CPUS=`grep processor /proc/cpuinfo | wc -l`
else
    CPUS=1
fi
/usr/bin/make -j`expr $CPUS + 1` "$@"

Answer 4

It appears that the MAKEFLAGS environment variable can pass flags that are part of every make run (at least for GNU make). I haven't had much luck with this myself, but it might be possible to use -l rather than -j to automatically run as many jobs as are appropriate for the number of cores you have available.

Answer 5

I would just put

alias make='make -j'

in ~/.profile or ~/.bashrc.

According to the manual:

If there is nothing looking like an integer after the ‘-j’ option, there is no limit on the number of job slots.

Answer 6

Until sometime in 2016, you could put this in your makefile: (GNU make tested)

MAKEFLAGS += "-j$(NUM_CORES) -l$(NUM_CORES)

(where NUM_PPROCS is calculated or set according to one of many of the other answers here) And, bam! you have multi-process building going on.

Given that this has stopped working, the best thing that I could come up with is this, where the makefile calls itself, but with -jX and -lX.

ifeq ($(PARALELL_WRAPPER_ABXCOEOEKCOEBMQJKHTOEUB),done)

all: ...
   ...

other_target: ...
    ...

else
# add parallelism equal to number of cores every time.
# "random" strings are to ensure uniqueness
NUM_CORES ?= $(shell grep -c "vendor_id" /proc/cpuinfo)
MAKEFLAGS +=" -j$(NUM_CORES) -l$(NUM_CORES) "

# for the default target case
parallel_wrapper_default_target_anthsqjkshbeohcbmeuthnoethoaeou:
    $(MAKE) PARALELL_WRAPPER_ABXCOEOEKCOEBMQJKHTOEUB=done

# catches everything else
% :
    $(MAKE) $@ PARALELL_WRAPPER_ABXCOEOEKCOEBMQJKHTOEUB=done

endif