[HDFS-3936] MiniDFSCluster shutdown races with BlocksMap usage

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0-alpha
• Fix Version/s: 2.0.3-alpha
• Hadoop Flags: Reviewed
• Assignee: Eli Collins
• Reporter: Eli Collins
• Created: 14/Sep/12 05:35
• Updated: 15/Feb/13 13:11
• Resolved: 18/Sep/12 17:09

Description

Looks like HDFS-3664 didn't fix the whole issue because the added join times out because the thread closing the BM (FSN#stopCommonServices) holds the FSN lock while closing the BM and the BM is block uninterruptedly trying to aquire the FSN lock.

2012-09-13 18:54:12,526 FATAL hdfs.MiniDFSCluster (MiniDFSCluster.java:shutdown(1355)) - Test resulted in an unexpected exit
org.apache.hadoop.util.ExitUtil$ExitException: Fatal exception with message null
stack trace
java.lang.NullPointerException
	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.getBlockCollection(BlocksMap.java:101)
	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeReplicationWorkForBlocks(BlockManager.java:1132)
	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeReplicationWork(BlockManager.java:1107)
	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeDatanodeWork(BlockManager.java:3061)
	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor.run(BlockManager.java:3023)
	at java.lang.Thread.run(Thread.java:662)

Issue Links

• is duplicated by: HDFS-3933 Unclean exit in ReplicationMonitor#run occasionally causes tests to fail
• relates to: HDFS-3937 The BlockManager should not use the FSN lock
• relates to: HDFS-3664 BlockManager race when stopping active services
• relates to: HDFS-3948 TestWebHDFS#testNamenodeRestart occasionally fails