What is the difference between ranks and processes in MPI?

Question

What is the difference between ranks and processes in MPI?

Answer 1

Here is the resource I learned all my MPI from, you might find it useful.

As to your question: processes are the actual instances of the program that are running. MPI allows you to create logical groups of processes, and in each group, a process is identified by its rank. This is an integer in the range [0, N-1] where N is the size of the group. Communicators are objects that handle communication between processes. An intra-communicator handles processes within a single group, while an inter-communicator handles communication between two distinct groups.

By default, you have a single group that contains all your processes, and the intra-communicator MPI_COMM_WORLD that handles communication between them. This is sufficient for most applications, and does blur the distinction between process and rank a bit. The main thing to remember is that the rank of a process is always relative to a group. If you were to split your processes into two groups (e.g. one group to read input and another group to process data), then each process would now have two ranks: the one it originally had in MPI_COMM_WORLD, and one in its new group.

Answer 2

Rank is a logical way of numbering processes. For instance, you might have 16 parallel processes running; if you query for the current process' rank via MPI_Comm_rank you'll get 0-15.

Rank is used to distinguish processes from one another. In basic applications you'll probably have a "master" process on rank = 0 that sends out messages to "slave" applications on rank 1-15. For more advanced applications you can divide workloads even further using ranks (i.e. 0 rank master process, 1-7 perform function A, 8-15 perform function B).

Answer 3

Every process that belongs to a communicator is uniquely identified by its rank. The rank of a process is an integer that ranges from zero up to the size of the communicator minus one. A process can determine its rank in a communicator by using the MPI_Comm_rank function that takes two arguments: the communicator and an integer variable rank:

int MPI_Comm_rank(MPI_Comm comm, int *rank) 

The parameter rank will store the rank of the process.

Note that each process that calls either one of these functions must belong in the supplied communicator, otherwise an error will occur.