[HDFS-5283] NN not coming out of startup safemode due to under construction blocks only inside snapshots also counted in safemode threshhold

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.1.1-beta, 3.0.0-alpha1
• Fix Version/s: 2.3.0
• Component/s: snapshots
• Hadoop Flags: Reviewed
• Assignee: Vinayakumar B
• Reporter: Vinayakumar B
• Created: 01/Oct/13 10:53
• Updated: 12/May/16 18:14
• Resolved: 16/Oct/13 18:40

Description

This is observed in one of our env:
1. A MR Job was running which has created some temporary files and was writing to them.
2. Snapshot was taken
3. And Job was killed and temporary files were deleted.
4. Namenode restarted.
5. After restart Namenode was in safemode waiting for blocks

Analysis
---------
1. Since the snapshot taken also includes the temporary files which were open, and later original files are deleted.
2. UnderConstruction blocks count was taken from leases. not considered the UC blocks only inside snapshots
3. So safemode threshold count was more and NN did not come out of safemode

Issue Links

• relates to: HDFS-5428 under construction files deletion after snapshot+checkpoint+nn restart leads nn safemode