[HDFS-5705] TestSecondaryNameNodeUpgrade#testChangeNsIDFails may fail due to ConcurrentModificationException

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha1
• Fix Version/s: 2.4.0
• Component/s: datanode
• Target Version/s: 2.4.0
• Hadoop Flags: Reviewed
• Assignee: Ted Yu
• Reporter: Ted Yu
• Created: 29/Dec/13 21:24
• Updated: 12/May/16 18:17
• Resolved: 03/Jan/14 18:30

Description

From https://builds.apache.org/job/Hadoop-Hdfs-trunk/1626/testReport/org.apache.hadoop.hdfs.server.namenode/TestSecondaryNameNodeUpgrade/testChangeNsIDFails/ :

java.util.ConcurrentModificationException: null
	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
	at java.util.HashMap$EntryIterator.next(HashMap.java:834)
	at java.util.HashMap$EntryIterator.next(HashMap.java:832)
	at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl.shutdown(FsVolumeImpl.java:251)
	at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeList.shutdown(FsVolumeList.java:218)
	at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.shutdown(FsDatasetImpl.java:1414)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.shutdown(DataNode.java:1309)
	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdownDataNodes(MiniDFSCluster.java:1464)
	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1439)
	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1423)
	at org.apache.hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade.doIt(TestSecondaryNameNodeUpgrade.java:97)
	at org.apache.hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade.testChangeNsIDFails(TestSecondaryNameNodeUpgrade.java:116)

The above happens when shutdown() is called in parallel to addBlockPool() or shutdownBlockPool().

Issue Links

• duplicates: HDFS-6098 ConcurrentModificationException exception during DataNode shutdown