Communication-aware message matching in MPI: Communication-aware message matching in MPI

The Message Passing Interface (MPI) is the de facto standard for parallel programming in High Performance Computing (HPC). Asynchronous communications in MPI involve message matching semantics that must be satisfied by the conforming libraries. The matching performance is in the critical path of communications in MPI. However, the current message matching approaches suffer from scalability issues and/or do not consider the message queue characteristics of the applications. In this paper, we propose a new message matching mechanism for MPI that can speed up the operation by allocating dedicated queues for certain communications of an application. More specifically, we propose a design that categorizes communications into a set of partners and non-partners based on the communication frequency in the corresponding queues. We propose a static and a dynamic approach for our message matching design. While the static approach works based on the information from a profiling stage, the dynamic approach utilizes the message queue characteristics at runtime. Our experimental evaluations show that the proposed design can provide up to 28x speedup in queue search time for long list traversals without degrading the performance for short list traversals. We can also gain up to 5x speedup for the FDS application, which is highly affected by the message matching performance.