Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8113

Add check for null BlockCollection pointers in BlockInfoContiguous structures

    Details

    • Target Version/s:

      Description

      The following copy constructor can throw NullPointerException if bc is null.

        protected BlockInfoContiguous(BlockInfoContiguous from) {
          this(from, from.bc.getBlockReplication());
          this.bc = from.bc;
        }
      

      We have observed that some DataNodes keeps failing doing block reports with NameNode. The stacktrace is as follows. Though we are not using the latest version, the problem still exists.

      2015-03-08 19:28:13,442 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: RemoteException in offerService
      org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException): java.lang.NullPointerException
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockInfo.(BlockInfo.java:80)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$BlockToMarkCorrupt.(BlockManager.java:1696)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.checkReplicaCorrupt(BlockManager.java:2185)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.processReportedBlock(BlockManager.java:2047)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.reportDiff(BlockManager.java:1950)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.processReport(BlockManager.java:1823)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.processReport(BlockManager.java:1750)
      at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.blockReport(NameNodeRpcServer.java:1069)
      at org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.blockReport(DatanodeProtocolServerSideTranslatorPB.java:152)
      at org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:26382)
      at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:587)
      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1026)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1623)
      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)

        Attachments

        1. HDFS-8113.02.patch
          2 kB
          Chengbing Liu
        2. HDFS-8113.patch
          0.8 kB
          Chengbing Liu

          Issue Links

            Activity

              People

              • Assignee:
                chengbing.liu Chengbing Liu
                Reporter:
                chengbing.liu Chengbing Liu
              • Votes:
                0 Vote for this issue
                Watchers:
                15 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: