Serialize mpi threads

Question

Is there anyway to serialize a certain part of your MPI code ? For example when printing the information out to the screen. Something like below:

MPI_SERIALI         


        

Answer 1

Nothing like that can be explicitly specified with MPI. You can, however use MPI_Gather to gather the stuff/values you want in one process and print them in order there.

Answer 2

I would not recommend to output anything on other nodes than the master node, as - depending on the platform that you are using - the slave nodes may not be able to handle output. Therefore, although it is nasty, you will have to collect all information to be printed out to the master node.

Answer 3

The way I do this in demonstration programs (and note - you would only do this in little demo programs because of the high synchronization cost; if you're trying to control output to a data file, you'd use MPI-IO, and if you're trying to coordinate output to the terminal, easiest to send data to task 0 and have it do all the output) is to loop over barriers, something like this:

#include <iostream>
#include <mpi.h>

using namespace std;

int main(int argc, char **argv) {

    int rank, size;
    int ierr;

    ierr = MPI_Init(&argc, &argv);

    ierr = MPI_Comm_size(MPI_COMM_WORLD, &size);
    ierr = MPI_Comm_rank(MPI_COMM_WORLD, &rank);

    for (int i=0; i<size; i++)
    {
        if (i == rank) {
            cout << "Hello from task " << rank << " of "
                 << size << " world!" << endl;
        }
        MPI_Barrier(MPI_COMM_WORLD);
    }

    MPI_Finalize();

    return 0;
}

(And as a smaller thing, MPI doesn't have threads, it has processes. That may seem like a small detail, but if you start combining MPI with OpenMP, for instance, the distinction between threads and processes becomes important.)