[HDFS-12638] Delete copy-on-truncate block along with the original block, when deleting a file being truncated

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.8.2
• Fix Version/s: 2.8.3, 2.7.5, 3.1.0, 2.10.0, 2.9.1, 3.0.1
• Component/s: hdfs
• Target Version/s: 2.8.3, 3.0.0, 2.9.1
• Hadoop Flags: Reviewed
• Assignee: Konstantin Shvachko
• Reporter: Jiandan Yang
• Created: 11/Oct/17 11:29
• Updated: 17/Jan/18 17:07
• Resolved: 01/Dec/17 03:36

Description

Active NamNode exit due to NPE, I can confirm that the BlockCollection passed in when creating ReplicationWork is null, but I do not know why BlockCollection is null, By view history I found HDFS-9754 remove judging whether BlockCollection is null.

NN logs are as following:

2017-10-11 16:29:06,161 ERROR [ReplicationMonitor] org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: ReplicationMonitor thread received Runtime exception.
java.lang.NullPointerException
        at org.apache.hadoop.hdfs.server.blockmanagement.ReplicationWork.chooseTargets(ReplicationWork.java:55)
        at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeReplicationWorkForBlocks(BlockManager.java:1532)
        at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeReplicationWork(BlockManager.java:1491)
        at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeDatanodeWork(BlockManager.java:3792)
        at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor.run(BlockManager.java:3744)
        at java.lang.Thread.run(Thread.java:834)

Issue Links

• is broken by: HDFS-9754 Avoid unnecessary getBlockCollection calls in BlockManager
• is related to: HDFS-7056 Snapshot support for truncate
• is related to: HDFS-8330 BlockInfoContiguous in blocksMap can have null BlockCollection
• is related to: HDFS-12880 Disallow abandoned blocks in the BlocksMap
• relates to: HDFS-13027 Handle possible NPEs due to deleted blocks in race condition
• relates to: HDFS-12884 BlockUnderConstructionFeature.truncateBlock should be of type BlockInfo