[HDFS-12272] chooseRemoteRack() semantics broken in trunk

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Invalid
• Affects Version/s: 3.0.0-alpha3
• Component/s: block placement
• Target Version/s: 3.0.0-beta1
• Assignee: Unassigned
• Reporter: Kihwal Lee
• Created: 07/Aug/17 19:57
• Updated: 07/Aug/17 20:02
• Resolved: 07/Aug/17 20:01

Description

The chooseRemoteRack() method in the default block placement policy was designed to pick from maximum number of racks. E.g. If asked to pick 2 and there are 2 or more racks available, the two will be on different racks. It wasn't implicit or accidental semantics. There was a specific logic in chooseRandom() that makes it happen.

This behavior is broken after HDFS-11530 as this logic was removed from chooseRandom(). Now the result is unpredictable. Sometimes the replicas end up in the same rack.

Issue Links

• is broken by: HDFS-11530 Use HDFS specific network topology to choose datanode in BlockPlacementPolicyDefault