[HDFS-12946] Add a tool to check rack configuration against EC policies

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.3.0, 3.2.2
• Component/s: erasure-coding
• Target Version/s: 3.3.0
• Assignee: Kitti Nanasi
• Reporter: Xiao Chen
• Created: 20/Dec/17 06:33
• Updated: 06/Sep/20 02:43
• Resolved: 03/Dec/18 18:04

Description

From testing we have seen setups with problematic racks / datanodes that would not suffice basic EC usages. These are usually found out only after the tests failed.

We should provide a way to check this beforehand.

Some scenarios:

• not enough datanodes compared to EC policy's highest data+parity number
• not enough racks to satisfy BPPRackFaultTolerant
• highly uneven racks to satisfy BPPRackFaultTolerant
• highly uneven racks (so that BPP's considerLoad logic may exclude some busy nodes on the rack, resulting in #2)