Automatic namespace recovery from the secondary image.

• Key: HADOOP-2585
• Type: New Feature
• Status: Closed
• Resolution: Fixed
• Priority: Major
• Assignee: Konstantin Shvachko
• Reporter: Konstantin Shvachko
• Affects Version/s: 0.16.0
• Fix Version/s: 0.18.0

Description

Hadoop has a three way (configuration controlled) protection from loosing the namespace image.

1. image can be replicated on different hard-drives of the same node;
2. image can be replicated on a nfs mounted drive on an independent node;
3. a stale replica of the image is created during periodic checkpointing and stored on the secondary name-node.

Currently during startup the name-node examines all configured storage directories, selects the
most up to date image, reads it, merges with the corresponding edits, and writes to the new image back
into all storage directories. Everything is done automatically.

If due to multiple hardware failures none of those images on mounted hard drives (local or remote)
are available the secondary image although stale (up to one hour old by default) can be still
used in order to recover the majority of the file system data.
Currently one can reconstruct a valid name-node image from the secondary one manually.
It would be nice to support an automatic recovery.