[HDFS-17477] IncrementalBlockReport race condition additional edge cases

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.3.5, 3.3.4, 3.3.6
• Component/s: auto-failover, ha, namenode
• Labels: pull-request-available
• Assignee: Danny Becker
• Reporter: Danny Becker
• Created: 17/Apr/24 22:18
• Updated: 24/Apr/24 18:20

Description

HDFS-17453 fixes a race condition between IncrementalBlockReports (IBR) and the Edit Log Tailer which can cause the Standby NameNode (SNN) to incorrectly mark blocks as corrupt when it transitions to Active. There are a few edge cases that HDFS-17453 does not cover.

For Example:
1. SNN1 loads the edits for b1gs1 and b1gs2.
2. DN1 reports b1gs1 to SNN1, so it gets queued for later processing.
3. DN1 reports b1gs2 to SNN1 so it gets added to the blocks map.
4. SNN1 transitions to Active (ANN1).
5. ANN1 processes the pending DN message queue and marks DN1->b1gs1 as corrupt because it was still in the queue.

Issue Links

• causes: HDFS-17499 removeQueuedBlock in PendingDataNodeMessages has memory leak
• links to: GitHub Pull Request #6748