How do I configure portable parallel builds in CMake?

问题

Is it somehow possible to be able to have a parallel build no matter which build tool is used?

Under Unix we can add make -jN where N are the number of threads, and under Windows I added to the CXX_FLAG "/MP" which is then used in Visual Studio to parallel build...(?) How can I make my version such that CMAKE_MAKE_PROGRAM is not always extended when I run CMake?

What is a general solution?

I came up with this:

# Add some multithreaded build support
MARK_AS_ADVANCED(MULTITHREADED_BUILD)
set(MULTITHREADED_BUILD 12 CACHE STRING "How many threads are used to build the project")
if(MULTITHREADED_BUILD)
    if(${CMAKE_GENERATOR} MATCHES "Unix Makefiles")
            message(STATUS ${CMAKE_BUILD_TOOL})
            set(CMAKE_MAKE_PROGRAM "${CMAKE_MAKE_PROGRAM} -j${MULTITHREADED_BUILD}")
            message(STATUS "Added arguments to CMAKE_BUILD_TOOL: ${CMAKE_MAKE_PROGRAM}")
    elseif(MSVC)
      set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /MP")
      message(STATUS "Added parallel build arguments to CMAKE_CXX_FLAGS: ${CMAKE_CXX_FLAGS}")
    endif()
endif()

回答1:

With CMake 3.12 this is possible. From the release notes:

The “cmake(1)” Build Tool Mode (“cmake –build”) gained “– parallel []” and “-j []” options to specify a parallel build level. They map to corresponding options of the native build tool.

As mentioned by dkg, you can also set the environment variable CMAKE_BUILD_PARALLEL_LEVEL.

Links to CMake's documentation:

Build a Project
CMAKE_BUILD_PARALLEL_LEVEL

回答2:

If you have CMake v2.8.8 or higher, you may use Ninja as an alternative of GNU make:

mkdir build
cd    build
cmake -G Ninja ..
ninja              # Parallel build (no need -j12)

mkdir build
cd    build
cmake -G Ninja ..
cmake --build .    # Parallel build using Ninja

As you can see, no need to use CMAKE_MAKE_PROGRAM, the build is run in parallel by default, optimizing the number of jobs depending on available CPU cores.

Ninja is based on a low-level JSON configuration to speed up the startup phase. Therefore its JSON configuration is not easy to write by hand, and I always generate it using a high-level tool/IDE:

CMake v2.8.8 (2012)
Qt Creator v2.6 (2012)
KDevelop v4.6 (2013)
Meson on Linux (2013)
... see the generators of Ninja configuration at https://github.com/ninja-build/ninja/wiki/List-of-generators-producing-ninja-build-files

As a C++ build often requires lots of memory, your computer must provide as much memory as the number of CPU cores.

As pointed out by Ruslan, CMake 3.12 (2018) has a new option cmake --build -j <N> to limit build to <N> cores (jobs) thus limiting the memory consumption (see also the documentation). If you use an older CMake version, you can still use cmake --build -- -j <N>. The option -- tells to CMake to pass the rest directly to the underlying builder tool, here it is Ninja.

回答3:

You can't do this cross-platform. The -jN option is a parameter to make, and not part of the generated Makefile. However, you could have CMake generate a Bash script that runs make for your project using -jN (where the script looks up the number of cores you have).

回答4:

I have settled down to writing a parallelmake.sh script for Unix Makefiles-based generators. This is done here: https://github.com/gabyx/ApproxMVBB

And the relevant parts in the the CMake file:

https://github.com/gabyx/ApproxMVBB/blob/master/CMakeLists.txt#L89

# Add some multithreaded build support =====================================================================================================
MARK_AS_ADVANCED(MULTITHREADED_BUILD)
SET(MULTITHREADED_BUILD ON CACHE BOOL "Parallel build with as many threads as possible!")
if(MULTITHREADED_BUILD)
    if(${CMAKE_GENERATOR} MATCHES "Unix Makefiles")
            file(COPY ${ApproxMVBB_ROOT_DIR}/cmake/parallelmake.sh DESTINATION ${PROJECT_BINARY_DIR}
                FILE_PERMISSIONS OWNER_READ OWNER_WRITE OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE
                NO_SOURCE_PERMISSIONS
            )
            SET(CMAKE_MAKE_PROGRAM "${PROJECT_BINARY_DIR}/parallelmake.sh")
            MESSAGE(STATUS "Set make program to ${PROJECT_BINARY_DIR}/parallelmake.sh")
    elseif(MSVC)
      SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}" "/MP")
      MESSAGE(STATUS "Added parallel build arguments to CMAKE_CXX_FLAGS: ${CMAKE_CXX_FLAGS}")
    endif()
endif()
# ========================================================================================================================================

来源：https://stackoverflow.com/questions/10688549/how-do-i-configure-portable-parallel-builds-in-cmake

标签

cmake

parallel-builds