[HDFS-12363] Possible NPE in BlockManager$StorageInfoDefragmenter#scanAndCompactStorages

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.6.0
• Fix Version/s: 3.0.0-beta1
• Component/s: namenode
• Target Version/s: 3.0.0-beta1
• Hadoop Flags: Reviewed
• Assignee: Xiao Chen
• Reporter: Xiao Chen
• Created: 28/Aug/17 05:34
• Updated: 02/Oct/19 17:15
• Resolved: 01/Sep/17 05:39

Description

Saw NN going down with NPE below:

ERROR org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Thread received Runtime exception.
java.lang.NullPointerException
at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$StorageInfoDefragmenter.scanAndCompactStorages(BlockManager.java:3897)
at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$StorageInfoDefragmenter.run(BlockManager.java:3852)
at java.lang.Thread.run(Thread.java:745)
2017-08-21 22:14:05,303 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1
2017-08-21 22:14:05,313 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: 

In that version, BlockManager code is:

3896  try {
3897           DatanodeStorageInfo storage = datanodeManager.
3898                 getDatanode(datanodesAndStorages.get(i)).
3899                getStorageInfo(datanodesAndStorages.get(i + 1));
3900            if (storage != null) {

Issue Links

• is broken by: HDFS-9260 Improve the performance and GC friendliness of NameNode startup and full block reports