[HDFS-3731] 2.0 release upgrade must handle blocks being written from 1.0

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.0.0-alpha
• Fix Version/s: 0.23.4, 2.0.2-alpha
• Component/s: datanode
• Target Version/s: 0.23.3
• Assignee: Kihwal Lee
• Reporter: Suresh Srinivas
• Created: 26/Jul/12 15:24
• Updated: 06/Nov/12 10:22
• Resolved: 27/Sep/12 18:23

Description

Release 2.0 upgrades must handle blocks being written to (bbw) files from 1.0 release. Problem reported by @brahmareddy
The DataNode will only have one block pool after upgrading from a 1.x release. (This is because in the 1.x releases, there were no block pools-- or equivalently, everything was in the same block pool). During the upgrade, we should hardlink the block files from the blocksBeingWritten directory into the rbw directory of this block pool. Similarly, on -finalize, we should delete the blocksBeingWritten directory.

Issue Links

• is blocked by: HDFS-3853 Port MiniDFSCluster enableManagedDfsDirsRedundancy option to branch-2
• is related to: HADOOP-8291 jenkins: include support for binary patches
• relates to: HBASE-6586 Quarantine Corrupted HFiles with hbck