[HDFS-5754] Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion - ASF JIRA

Voters

Watch issue

Watchers

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Sub-task
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: HDFS-5535 (Rolling upgrades)
Component/s: datanode, namenode
Labels:
None

Hadoop Flags:

Reviewed

Description

Currently, LayoutVersion defines the on-disk data format and supported features of the entire cluster including NN and DNs. LayoutVersion is persisted in both NN and DNs. When a NN/DN starts up, it checks its supported LayoutVersion against the on-disk LayoutVersion. Also, a DN with a different LayoutVersion than NN cannot register with the NN.

We propose to split LayoutVersion into two independent values that are local to the nodes:

NamenodeLayoutVersion - defines the on-disk data format in NN, including the format of FSImage, editlog and the directory structure.
DatanodeLayoutVersion - defines the on-disk data format in DN, including the format of block data file, metadata file, block pool layout, and the directory structure.

The LayoutVersion check will be removed in DN registration. If NamenodeLayoutVersion or DatanodeLayoutVersion is changed in a rolling upgrade, then only rollback is supported and downgrade is not.