Mpisee: communicator-centric profiling of MPI applications

Abstract

mpisee is a lightweight profiling tool designed to track MPI communication operations per communicator, providing fine-grained insights into MPI applications that use communicators to partition MPI communication. While existing profiling tools offer valuable information, they may limit detailed analysis and optimization for such MPI applications, as they do not associate MPI communication with their communicator. Additionally, mpisee categorizes MPI communication operations based on message size, offering more granular information. It uses an SQLite database to efficiently store the profiling data, enabling users to analyze the application's profile from various perspectives, focusing on specific MPI ranks, operations, and more. Our analysis shows that mpisee incurs less than 5% overhead, performing on par with other state-of-the-art profilers. We demonstrate mpisee 's effectiveness by profiling and analyzing an FFT application, revealing potential performance bottlenecks related to the MPI_Alltoallv collective operation on small communicators and insights not available by other profilers. Leveraging this detailed information, we improved the application's overall performance by selecting different algorithms for MPI_Alltoallv and measuring their performance on different communicators with mpisee. This study illustrates mpisee 's utility and highlights the significant advantages of a communicator-centric approach in MPI profiling.