Processor/socket affinity in openMPI?

Question

I know,there are some basic function in openMPI implementation for mapping the different processes to different cores of different sockets(if the system have more than one socke

Answer 1

1. If there is equal communication between each node and the root and no other communication pattern, then communication will not influence the performance of a specific process->socket mapping. (This is assuming a regular symmetric interconnect topology between the sockets.) Otherwise you usually try to place process pairs with heavy communication close to each other in the communication topology. With MPI on shared memory systems that may not be relevant, but on clusters it certainly is.

However load balancing may also have an effect on the performance of the mapping. If some processes wait for a message/barrier, the other cores on that socket may be able to utilize a higher turbo frequency. This heavily depends on the runtime behavior of the application. An application only consisting of sleep and transfer does not really make sense.

1. You can use libnuma / sched_getaffinity to confirm your process pinning manually.

There are a number of performance analysis tools that would be helpful to answer your questions. For example OpenMPI comes with VampirTrace that produces a trace containing information about the MPI communication and more. You can view with Vampir.