[HDFS-12299] Race Between update pipeline and DN Re-Registration

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 2.9.0, 3.0.0-beta1, 2.8.2, 2.7.6
• Target Version/s: 2.7.6
• Hadoop Flags: Reviewed
• Assignee: Brahma Reddy Battula
• Reporter: Brahma Reddy Battula
• Created: 14/Aug/17 13:04
• Updated: 02/Oct/19 17:15
• Resolved: 25/Aug/17 20:06

Description

Scenario

• Started pipeline with DN1->DN2->DN3
• DN1 is re-reg and update pipeline is called
• Update pipeline will success with DN1->DN3->DN4
• Again update pipeline is called,which will fail with NPE.

In step3 updatepipeline will set the storages as null since DN1 re-reg(which will remove and add the storages)
FSNamesystem#updatePipelineInternal

   lastBlock.getUnderConstructionFeature().setExpectedLocations(lastBlock,
        storages, lastBlock.getBlockType())

Issue Links

• breaks: HDFS-12364 [branch-2.8.2] Fix the Compile Error after HDFS-12299
• is related to: HDFS-13486 Backport HDFS-11817 (A faulty node can cause a lease leak and NPE on accessing data) to branch-2.7