Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7285 Erasure Coding Support inside HDFS
  3. HDFS-8827

Erasure Coding: Fix NPE when NameNode processes over-replicated striped blocks

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: HDFS-7285
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In our test cluster, when namenode processed over replicated striped blocks, null pointer exception(NPE) occurred. This happened under below situation: 1) some datanodes shutdown. 2) namenode recovers block group which lost internal blocks. 3) restart the stopped datanodes. 4) namenode processes over replicated striped blocks. 5) NPE occurs
      I think BlockPlacementPolicyDefault#chooseReplicaToDelete will return null in this situation which causes this NPE problem.

        Attachments

        1. processing-over-replica-npe.log
          3 kB
          Takuya Fukudome
        2. HDFS-8827-HDFS-7285.05.patch
          11 kB
          Walter Su
        3. HDFS-8827-HDFS-7285.04.patch
          11 kB
          Walter Su
        4. HDFS-8827.3.patch
          6 kB
          Takuya Fukudome
        5. HDFS-8827.2.patch
          4 kB
          Jing Zhao
        6. HDFS-8827.1.patch
          2 kB
          Takuya Fukudome

          Activity

            People

            • Assignee:
              walter.k.su Walter Su
              Reporter:
              tfukudom Takuya Fukudome
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: