maptoreduce

MapReduce is a programming model and framework for processing large data sets on a distributed cluster. A MapReduce program runs two functions: Map and Reduce. The Map function processes input records and emits intermediate key-value pairs. The framework then groups these pairs by key and passes the list of values for each key to the Reduce function, which produces the final output.

The model was introduced by Google in 2004 in a paper by Jeffrey Dean and Sanjay Ghemawat. It gained widespread adoption in open-source form through the Apache Hadoop project, which began in 2006 and provided a scalable, fault-tolerant implementation for running MapReduce jobs on commodity hardware. MapReduce became a foundational approach for batch data processing in many organizations and data centers.

In a typical deployment, data is split into blocks and processed in parallel by Map tasks across a cluster. Each Map task emits key-value pairs; a partitioner routes keys to corresponding Reduce tasks. The framework performs a shuffle and sort step to collate all values for each key, after which the Reduce tasks execute and write output to a distributed file system. The model supports optional optimizations such as combiner functions and offers fault tolerance through task re-execution.

MapReduce is well suited to batch processing tasks like log analysis, indexing, and large-scale data transformations. While newer engines (such as Apache Spark) provide faster performance for iterative workloads and in-memory processing, MapReduce remains a robust and widely used approach for simple, scalable batch jobs in many data ecosystems. Some references may spell the concept as map-to-reduce or map reduce, but MapReduce is the standard designation.