[HDFS-15288] Add Available Space Rack Fault Tolerant BPP

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.2, 3.4.0
• Fix Version/s: 3.4.0, 3.3.2
• Component/s: block placement
• Hadoop Flags: Reviewed

Release Note

Added a new BlockPlacementPolicy: "AvailableSpaceRackFaultTolerantBlockPlacementPolicy" which uses the same optimization logic as the AvailableSpaceBlockPlacementPolicy along with spreading the replicas across maximum number of racks, similar to BlockPlacementPolicyRackFaultTolerant.
The BPP can be configured by setting the blockplacement policy class as org.apache.hadoop.hdfs.server.blockmanagement.AvailableSpaceRackFaultTolerantBlockPlacementPolicy

• Assignee: Ayush Saxena
• Reporter: Ayush Saxena
• Created: 20/Apr/20 18:40
• Updated: 11/Feb/24 03:58
• Resolved: 23/May/20 13:06

Description

The Present AvailableSpaceBlockPlacementPolicy extends the Default Block Placement policy, which makes it apt for Replicated files. But not very efficient for EC files, which by default use. BlockPlacementPolicyRackFaultTolerant. So propose a to add new BPP having similar optimization as ASBPP where as keeping the spread of Blocks to max racks, i.e as RackFaultTolerantBPP.

This could extend BlockPlacementPolicyRackFaultTolerant, rather than the BlockPlacementPOlicyDefault like ASBPP and keep other logics of optimization same as ASBPP

Issue Links

• duplicates: HDFS-15511 Support AvailableSpaceBlockPlacementPolicy in BlockPlacementPolicyRackFaultTolerant