[HDFS-17467] IncrementalBlockReportManager#getPerStorageIBR may throw NPE when remove volumes

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.4.0
• Component/s: datanode
• Labels: pull-request-available
• Assignee: farmmamba Assign to me
• Reporter: farmmamba
• Created: 15/Apr/24 08:22
• Updated: 24/Apr/24 02:02

Description

When we remove volumes, it may cause IncrementalBlockReportManager#getPerStorageIBR throws NPE.

Consider below situation:

1、we have down createRbw、finalizeBlock.  But have not done datanode.closeBlock in method `BlockReceiver.PacketResponder#finalizeBlock`.

2、we remove volume which replica was written to and it executes such code: `storageMap.remove(storageUuid);`

3、 we begin to execute datanode.closeBlock which try to send IBR to NameNode. but when getting DatanodeStorage from storageMap using 

storageUuid, we will get null because we have remove this storageUuid key from storageMap.

4、Throw NPE in getPerStorageIBR method, because ConcurrentHashMap don't allow null key.

 

 

 

Issue Links

• links to: GitHub Pull Request #6730