Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9313

Possible NullPointerException in BlockManager if no excess replica can be chosen

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.3, 2.6.4, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      HDFS-8647 makes it easier to reason about various block placement scenarios. Here is one possible case where BlockManager won't be able to find the excess replica to delete: when storage policy changes around the same time balancer moves the block. When this happens, it will cause NullPointerException.

      java.lang.NullPointerException
      	at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy.adjustSetsWithChosenReplica(BlockPlacementPolicy.java:156)
      	at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseReplicasToDelete(BlockPlacementPolicyDefault.java:978)
      

      Note that it isn't found in any production clusters. Instead, it is found from new unit tests. In addition, the issue has been there before HDFS-8647.

        Attachments

        1. HDFS-9313.branch27.patch
          4 kB
          Xiao Chen
        2. HDFS-9313.branch26.patch
          4 kB
          Xiao Chen
        3. HDFS-9313-2.patch
          5 kB
          Ming Ma
        4. HDFS-9313.patch
          5 kB
          Ming Ma

          Issue Links

            Activity

              People

              • Assignee:
                mingma Ming Ma
                Reporter:
                mingma Ming Ma
              • Votes:
                0 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: