[HDFS-14578] AvailableSpaceBlockPlacementPolicy always prefers local node

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.8.0, 2.7.4, 3.0.0-alpha1
• Fix Version/s: 3.3.0, 3.1.4, 3.2.2
• Component/s: block placement
• Assignee: Ayush Saxena
• Reporter: Wei-Chiu Chuang
• Created: 17/Jun/19 22:59
• Updated: 02/Apr/20 01:09
• Resolved: 11/Jan/20 05:03

Description

It looks like AvailableSpaceBlockPlacementPolicy prefers local disk just like in the BlockPlacementPolicyDefault

 

As Yongjun mentioned in HDFS-8131, 

Class AvailableSpaceBlockPlacementPolicy extends BlockPlacementPolicyDefault. But it doesn't change the behavior of choosing the first node in BlockPlacementPolicyDefault, so even with this new feature, the local DN is always chosen as the first DN (of course when it is not excluded), and the new feature only changes the selection of the rest of the two DNs.

I'm file this Jira as I groom Cloudera's internal Jira and found this unreported issue. We do have a customer hitting this problem. I don't have a fix, but thought it would be beneficial to report it to Apache Jira.