Prototyping the Reference Implementation of Persistent Non-Blocking Collective Communication in MPI
Type of DegreeMaster's Thesis
Computer Science and Software Engineering
MetadataShow full item record
This research focuses on the design, prototype implementation and standardization sta- tus of persistent non-blocking collective operations for the Message Passing Interface (MPI) standard, through functional extension of OpenMPI, an open-source MPI implementation. MPI is an integral tool in the field of parallel computing, providing fundamental communi- cation techniques for large scale computations. Optimization techniques such as persistence (planned transfers) and non-blocking operations (overlapping communication and compu- tation) have been present in point-to-point communication since MPI-1. Similarly, collec- tive operations (communication patterns among groups of processes) have been historically available as a tool to promote convenience and efficiency. More recently, the adoption of non-blocking collective operations into the MPI standard has offered further performance opportunity. For data-parallel and regular computations with fixed communication pat- terns, more optimization potential can be revealed through the application of persistence to these non-blocking collective functions, yet to be included in the MPI standard. Persistent operations allow MPI implementations to make intelligent choices about al- gorithm and resource utilization once, and amortize the decision cost across many uses in a long-running program. This research presents the first prototype implementation of per- sistence applied to non-blocking collective operations, and demonstrates the functionailty of the corresponding APIS. Early performance results of this implementation and examples illustrating the potential performance enhancements for such operations are presented. Fur- ther enhancement of the current implementation prototype, and additional opportunities to enhance performance through the application of these new APIs comprise future work.