[HDFS-9535] Newly completed blocks in IBR should not be considered under-replicated too quickly

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.8.0
• Fix Version/s: 2.8.0, 3.0.0-alpha1
• Component/s: namenode
• Hadoop Flags: Reviewed
• Assignee: Mingliang Liu
• Reporter: Jing Zhao
• Created: 10/Dec/15 00:42
• Updated: 30/Aug/16 01:22
• Resolved: 14/Dec/15 19:24

Description

TestReplication#testNoExtraReplicationWhenBlockReceivedIsLate failed in several Jenkins run (e.g., https://builds.apache.org/job/PreCommit-HDFS-Build/13818/testReport/). The failure is on the last assertNoReplicationWasPerformed check.

This test failure reveals a scenario that HDFS-1172 missed: if a block is first committed by the client, and then the first IBR comes to the NN, as proposed by HDFS-1172, we should still put the remaining expected replicas into the pending queue, instead of the under-replicated queue. Please see liuml07's comment here for more details.

Issue Links

• is related to: HDFS-1172 Blocks in newly completed files are considered under-replicated too quickly