[HDFS-1073] Simpler model for Namenode's fs Image and edit Logs - ASF JIRA

XML

Word

Printable

JSON

Hadoop Flags:

Incompatible change, Reviewed
Release Note:

Hide
The NameNode's storage layout for its name directories has been reorganized to be more robust. Each edit now has a unique transaction ID, and each file is associated with a transaction ID (for checkpoints) or a range of transaction IDs (for edit logs).

Show
The NameNode's storage layout for its name directories has been reorganized to be more robust. Each edit now has a unique transaction ID, and each file is associated with a transaction ID (for checkpoints) or a range of transaction IDs (for edit logs).

The naming and handling of NN's fsImage and edit logs can be significantly improved resulting simpler and more robust code.

is related to

HDFS-955 FSImage.saveFSImage can lose edits

HDFS-14378 Simplify the design of multiple NN and both logic of edit log roll and checkpoint

1.	1073: consider adding END_LOG_SEGMENT txn when finalizing inprogress logs at startup	Open	Unassigned
2.	1073: Fault injection for StorageDirectory failures during read/write of FSImage/Edits files	Open	Unassigned
3.	Complete decoupling of failure states between edits and image dirs	Open	Unassigned