Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11827

NPE is thrown when log level changed in BlockPlacementPolicyDefault#chooseRandom() method

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.6.0
    • 2.9.0, 3.0.0-alpha4
    • None
    • None
    • hadoop-version: 2.6.0-cdh5.4.4

    • Reviewed

    Description

      This issue was found by my colleague when changing log-level of BlockPlacementPolicy using "hadoop daemonlog" command.

      The exception stack trace is below:

      2017-05-16 13:36:51,053 ERROR org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: ReplicationMonitor thread received Runtime exception.
      java.lang.NullPointerException
              at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseRandom(BlockPlacementPolicyDefault.java:666)
              at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseRemoteRack(BlockPlacementPolicyDefault.java:607)
              at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:375)
              at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:446)
              at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:446)
              at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:240)
              at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:119)
              at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyWithMultiDC.chooseTargetInternal(BlockPlacementPolicyWithMultiDC.java:263)
              at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyWithMultiDC.chooseTarget(BlockPlacementPolicyWithMultiDC.java:214)
              at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationWork.chooseTargets(BlockManager.java:3836)
              at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationWork.access$200(BlockManager.java:3801)
              at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeReplicationWorkForBlocks(BlockManager.java:1394)
              at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeReplicationWork(BlockManager.java:1300)
              at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeDatanodeWork(BlockManager.java:3764)
              at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor.run(BlockManager.java:3711)
              at java.lang.Thread.run(Thread.java:745)
      

      After checking BlockPlacementPolicyDefault code of trunk, I found that BlockPlacementPolicyDefault class missed some NPE check code in chooseRandom() method.

      Attachments

        1. HDFS-11827.001.patch
          1 kB
          xupeng

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            xupener xupeng
            xupener xupeng
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment