MPI

Message Passing Interface (MPI) is a standardized and portable API used for writing parallel programs that run on distributed-memory systems and, with extensions, shared-memory architectures. MPI provides mechanisms for processes to communicate by sending and receiving messages, and to synchronize their execution. It is designed for high-performance computing applications and is implemented in several language bindings, most commonly C, C++, and Fortran.

MPI was developed in the early 1990s by the MPI Forum as a consensus standard to replace ad hoc libraries. MPI-1, released in 1994, defined basic point-to-point communication and collective operations. MPI-2 added dynamic process management, parallel I/O, and one-sided communication. MPI-3 introduced nonblocking collectives, improved one-sided communication, and enhanced derived data types. The MPI-4 standard is the current major revision, extending usability and performance features and addressing modern hardware trends.

Program structure in MPI revolves around processes that participate in one or more communicators. Each process has a rank, and communications occur within a communicator (MPI_Comm) among a group of ranks. Typical entry points are MPI_Init and MPI_Finalize. Point-to-point operations include blocking and nonblocking sends and receives (e.g., MPI_Send, MPI_Recv, MPI_Isend, MPI_Irecv), while collective operations involve all processes in a communicator (e.g., MPI_Bcast, MPI_Scatter, MPI_Gather, MPI_Allreduce). MPI also supports derived datatypes to describe noncontiguous data layouts.

Beyond basic messaging, MPI offers dynamic process management (spawning new processes), one-sided communication with remote memory access (RMA), topologies (Cartesian and graph), and parallel I/O interfaces. It is designed to be portable across vendors and architectures and to provide high performance by enabling implementation-specific optimizations.

Numerous implementations exist, including MPICH, Open MPI, MVAPICH, Intel MPI, and vendor-specific distributions (e.g., Cray MPI). MPI remains the de facto standard for building scalable, portable parallel applications in scientific computing, engineering simulations, and data-intensive workloads.