[HDFS-11293] [SPS]: Local DN should be given preference as source node, when target available in same node

• Type: Sub-task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: HDFS-10285
• Fix Version/s: HDFS-10285, 3.2.0
• Component/s: namenode
• Hadoop Flags: Reviewed
• Assignee: Uma Maheswara Rao G
• Reporter: Yuanbo Liu
• Created: 05/Jan/17 07:23
• Updated: 13/Aug/18 05:39
• Resolved: 09/Jan/17 22:40

Description

In FsDatasetImpl#createTemporary, we use volumeMap to get replica info by block pool id. But in this situation:

datanode A => {DISK, SSD}, datanode B => {DISK, ARCHIVE}.
1. the same block replica exists in A[DISK] and B[DISK].
2. the block pool id of datanode A and datanode B are the same.

Then we start to change the file's storage policy and move the block replica in the cluster. Very likely we have to move block from B[DISK] to A[SSD], at this time, datanode A throws ReplicaAlreadyExistsException.

Issue Links

• is related to: HDFS-11284 [SPS]: fix issue of moving blocks with satisfier while changing replication factor