[HADOOP-2585] Automatic namespace recovery from the secondary image.

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.16.0
• Fix Version/s: 0.18.0
• Hadoop Flags: Incompatible change, Reviewed

Release Note

Improved management of replicas of the name space image. If all replicas on the Name Node are lost, the latest check point can be loaded from the secondary Name Node. Use parameter "-importCheckpoint" and specify the location with "fs.checkpoint.dir." The directory structure on the secondary Name Node has changed to match the primary Name Node.

• Assignee: Konstantin Shvachko
• Reporter: Konstantin Shvachko
• Created: 12/Jan/08 00:22
• Updated: 08/Jul/09 16:42
• Resolved: 11/Apr/08 21:19

Description

Hadoop has a three way (configuration controlled) protection from loosing the namespace image.

1. image can be replicated on different hard-drives of the same node;
2. image can be replicated on a nfs mounted drive on an independent node;
3. a stale replica of the image is created during periodic checkpointing and stored on the secondary name-node.

Currently during startup the name-node examines all configured storage directories, selects the
most up to date image, reads it, merges with the corresponding edits, and writes to the new image back
into all storage directories. Everything is done automatically.

If due to multiple hardware failures none of those images on mounted hard drives (local or remote)
are available the secondary image although stale (up to one hour old by default) can be still
used in order to recover the majority of the file system data.
Currently one can reconstruct a valid name-node image from the secondary one manually.
It would be nice to support an automatic recovery.

Issue Links

• incorporates: HADOOP-3069 A failure on SecondaryNameNode truncates the primary NameNode image.
• incorporates: HADOOP-2987 Keep two generations of fsimage