namenode

A NameNode is the master server in the Hadoop Distributed File System (HDFS) that manages the filesystem namespace and metadata. It keeps the directory tree of all files and directories, along with file permissions, timestamps, and the mapping of files to the blocks stored on DataNodes. The actual file data are stored on DataNodes; the NameNode does not store user data.

The NameNode maintains metadata in memory for fast access and persists it on disk as two main structures: an image file (fsimage) that captures the filesystem state and an edit log that records every change to the namespace. On startup, the NameNode reads the latest fsimage and replays the edits to reconstruct the current namespace. Each namespace operation, such as create, delete, or rename, is appended to the edit log to ensure durability. A periodic checkpoint can consolidate the namespace by applying edits to a new fsimage. In older deployments, a SecondaryNameNode performed this task; modern clusters use a Standby NameNode for high availability.

High availability: A single NameNode represents a potential single point of failure. To mitigate this, many deployments run an active NameNode with a standby NameNode and a set of journal nodes to synchronize metadata (JournalNode-based quorum) or use a Quorum Journal Manager. When the active NameNode fails, the standby can take over with minimal disruption. The NameNode also handles communication with DataNodes through heartbeats and block reports, coordinating block placement, replication, deletion, and redirection of clients to the appropriate DataNodes. It exposes filesystem metadata to clients for operations like open, read, write, and list status.