The Message Passing Interface MPI is the most commonly used programming standard for parallel applications. To support such applications, the DEEP software stack includes the fully standard compliant MPI-2 implementation Parastation MPI by ParTec which has been modified to reflect the peculiarities of the DEEP system.
The actual offloading of highly-scalable code parts shall be as close to existing standards as possible. Therefore the dynamic process model of MPI-2, namely MPI_Comm_spawn(), is used for DEEP’s offloading mechanism. Besides spawning processes on the Booster Nodes it allows for an efficient exchange of data between Cluster Nodes and Booster Nodes with MPI semantics.
MPI_Comm_spawn is a collective operation performed by a subset of the processes of an application started on the Cluster. An inter-communicator is returned, providing a connection handle to the children. Each child has to call MPI_Init, as usual, and can get access to the inter-communicator via MPI_Get_parent.
An inter-communicator as defined by the MPI standard contains two groups of processes and naturally allows point-to-point communication between a member of one group and a member of the other group. Starting with MPI-2, collective operations have been extended and defined for inter-communicators.
DEEP’s programming model is based on MPI for intra-Booster as well as for intra-Cluster communications. Together with the offloading mechanism and Cluster-Booster communication they form a global MPI, i.e. a heterogeneous MPI implementation that is usable on all node types and allows for communication among Cluster Nodes and Booster Nodes, respectively, and at the same time between the Cluster and the Booster parts of the system.