[HDFS-4884] Provide a way to enforce new Block Placement Policy on existing data after switching from old one

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.2.0, 2.0.4-alpha
• Assignee: Junping Du
• Reporter: Junping Du
• Created: 06/Jun/13 14:46
• Updated: 21/Jan/16 09:03

Description

The BlockPlacementPolicy (BPP) is extensible and already multiple instances in system: BlockPlacementPolicyDefault, BlockPlacementPolicyWithNodeGroup, BlockPlacementPolicyRAID, etc. When cluster is switched from one BPP to another BPP, HDFS will not check if block locations conform the new BPP. Now, you can manually fsck on specific directory to identify replica placement policy is violated for some blocks but have no way to fix it so far. We should provide way to fix it.

Issue Links

• is related to: HDFS-4894 Multiple Replica Placement Policies support in one HDFS cluster

Sub-Tasks

• Update verifyBlockPlacement() API in BlockPlacementPolicy: Closed
• Override verifyBlockPlacement() API in BlockPlacementPolicyWithNodeGroup: Resolved
• Add a function of "-fixmisplaced" to fsck for fixing blocks in mistaken placed: Open