This lesson is being piloted (Beta version)

Non-blocking Communication

Overview

Teaching: 10 min
Exercises: 20 min
Questions
  • How do I interleave communication and computation?

Objectives
  • Introduce MPI_Isend, MPI_Irecv, MPI_Test and MPI_Wait.

Display Language  

Non-Blocking Communication

Send and Receive

In one of the previous lessons we used the MPI_Send and MPI_Recv functions to communicate between the ranks. We saw that these functions are blocking: MPI_Send will only return when the program can safely modify the send buffer and MPI_Recv will only return once the data has been received and written to the receive buffer. This is safe and usually straightforward, but causes the program to wait while the communication is happening. Usually there is computation that we could perform while waiting for data.

The MPI standard includes non-blocking versions of the send and receive functions, MPI_Isend and MPI_Irecv. These function will return immediately, giving you more control of the flow of the program. After calling them, it is not safe to modify the sending or the receiving buffer, but the program is free to continue with other operations. When it needs the data in the buffers, it needs to make sure the communication process is complete using the MPI_Wait and MPI_Test functions.

There’s one new parameter here, a request. This is used to keep track of each separate transfer started by the program. You can use it to check the status of a transfer using the MPI_Test function, or call MPI_Wait to wait until the transfer is complete.

Test and Wait

MPI_Test will return the status of the transfer specified by a request and MPI_Wait will wait until the transfer is complete before returning. The request can be created by either MPI_Isend or MPI_Irecv.

Examples

These functions can be used similarly to MPI_Send and MPI_Recv. Here is how you could replace MPI_Send and MPI_Recv in the program that sends the “Hello World!” string by MPI_ISend, MPI_IRecv and MPI_Wait:

Non-Blocking Communication

Here is the blocking example again. Fix the problem using MPI_Isend, MPI_Irecv and MPI_Wait.

Key Points

  • Non-blocking functions allows interleaving communication and computation.