Prototyping the Reference Implementation of Persistent Non-Blocking Collective Communication in MPI
Morgan, Ian Bradley
Type of DegreeMaster's Thesis
DepartmentComputer Science and Software Engineering
MetadataShow full item record
This research focuses on the design, prototype implementation and standardization sta- tus of persistent non-blocking collective operations for the Message Passing Interface (MPI) standard, through functional extension of OpenMPI, an open-source MPI implementation. MPI is an integral tool in the field of parallel computing, providing fundamental communi- cation techniques for large scale computations. Optimization techniques such as persistence (planned transfers) and non-blocking operations (overlapping communication and compu- tation) have been present in point-to-point communication since MPI-1. Similarly, collec- tive operations (communication patterns among groups of processes) have been historically available as a tool to promote convenience and efficiency. More recently, the adoption of non-blocking collective operations into the MPI standard has offered further performance opportunity. For data-parallel and regular computations with fixed communication pat- terns, more optimization potential can be revealed through the application of persistence to these non-blocking collective functions, yet to be included in the MPI standard. Persistent operations allow MPI implementations to make intelligent choices about al- gorithm and resource utilization once, and amortize the decision cost across many uses in a long-running program. This research presents the first prototype implementation of per- sistence applied to non-blocking collective operations, and demonstrates the functionailty of the corresponding APIS. Early performance results of this implementation and examples illustrating the potential performance enhancements for such operations are presented. Fur- ther enhancement of the current implementation prototype, and additional opportunities to enhance performance through the application of these new APIs comprise future work.