What is the difference between isend and issend?

Question

Need clarification to my understanding of isend and issend as given in Send Types

My understanding is that isend will return once the send buffer is free, i.e. when

Answer 1

Both MPI_Isend() and MPI_Issend() return immediately, but in both cases you can't use the send buffer immediately.

Think of the difference that there is between MPI_Send() and MPI_Ssend():

• MPI_Send() can be buffered or it can be synchronous if the buffer is too large to be buffered locally, and in this case it waits to complete sending the data to the corresponding receive operation.

• MPI_Ssend() is always synchronous: it always waits to complete sending the data to the corresponding receive operation.

The inner working of the corresponding "I"-operations is very similar, except for the fact that they both don't block (return immediately): the difference is only when the MPI library signals to the user program that you can use the send-buffer (that is: MPI_Wait() returns or MPI_Test() returns true - the so called send-complete operation of the non-blocking send):

• with MPI_Isend() this can happen either when the data has been copied locally in a buffer owned by the MPI library, if below the "synchronous threshold", or when the data has been actually moved to the sibling task: the send-complete operation can be local, in case the underlying send operation is buffered.

• With MPI_Issend() MPI doesn't ever buffer data locally and the "buffer-free condition" is returned only after the data has been actually transferred (and probably ack'ed, at low level): the send-complete operation is non-local.

The MPI standard document is quite pedantic on these aspects. See section 3.7 Nonblocking Communication.