[HDFS-9313] Possible NullPointerException in BlockManager if no excess replica can be chosen

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.8.0, 2.7.3, 2.6.4, 3.0.0-alpha1
• Target Version/s: 2.6.4
• Hadoop Flags: Reviewed
• Assignee: Ming Ma
• Reporter: Ming Ma
• Created: 27/Oct/15 00:25
• Updated: 30/Aug/16 01:24
• Resolved: 03/Nov/15 03:42

Description

HDFS-8647 makes it easier to reason about various block placement scenarios. Here is one possible case where BlockManager won't be able to find the excess replica to delete: when storage policy changes around the same time balancer moves the block. When this happens, it will cause NullPointerException.

java.lang.NullPointerException
	at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy.adjustSetsWithChosenReplica(BlockPlacementPolicy.java:156)
	at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseReplicasToDelete(BlockPlacementPolicyDefault.java:978)

Note that it isn't found in any production clusters. Instead, it is found from new unit tests. In addition, the issue has been there before HDFS-8647.

Issue Links

• is duplicated by: HDFS-8770 ReplicationMonitor thread received Runtime exception: NullPointerException when BlockManager.chooseExcessReplicates
• is related to: HDFS-9440 Improve BlockPlacementPolicyDefault's to be able to rebalance when picking of excess replicas
• is related to: HDFS-9314 Improve BlockPlacementPolicyDefault's picking of excess replicas