[HDFS-15796] ConcurrentModificationException error happens on NameNode occasionally

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.1.1
• Fix Version/s: 3.4.0, 3.2.3, 3.3.2
• Component/s: hdfs
• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Assignee: Daniel Ma
• Reporter: Daniel Ma
• Created: 28/Jan/21 06:23
• Updated: 27/Jan/24 02:55
• Resolved: 09/Jul/21 09:50

Description

ConcurrentModificationException error happens on NameNode occasionally.

 

2021-01-23 20:21:18,107 | ERROR | RedundancyMonitor | RedundancyMonitor thread received Runtime exception.  | BlockManager.java:4746
java.util.ConcurrentModificationException
	at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:909)
	at java.util.ArrayList$Itr.next(ArrayList.java:859)
	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeReconstructionWorkForBlocks(BlockManager.java:1907)
	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeBlockReconstructionWork(BlockManager.java:1859)
	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeDatanodeWork(BlockManager.java:4862)
	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$RedundancyMonitor.run(BlockManager.java:4729)
	at java.lang.Thread.run(Thread.java:748)

 

 

Issue Links

• is caused by: HDFS-15159 Prevent adding same DN multiple times in PendingReconstructionBlocks