Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7285 Erasure Coding Support inside HDFS
  3. HDFS-8827

Erasure Coding: Fix NPE when NameNode processes over-replicated striped blocks

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • HDFS-7285
    • None
    • None
    • Reviewed

    Description

      In our test cluster, when namenode processed over replicated striped blocks, null pointer exception(NPE) occurred. This happened under below situation: 1) some datanodes shutdown. 2) namenode recovers block group which lost internal blocks. 3) restart the stopped datanodes. 4) namenode processes over replicated striped blocks. 5) NPE occurs
      I think BlockPlacementPolicyDefault#chooseReplicaToDelete will return null in this situation which causes this NPE problem.

      Attachments

        1. processing-over-replica-npe.log
          3 kB
          Takuya Fukudome
        2. HDFS-8827-HDFS-7285.05.patch
          11 kB
          Walter Su
        3. HDFS-8827-HDFS-7285.04.patch
          11 kB
          Walter Su
        4. HDFS-8827.3.patch
          6 kB
          Takuya Fukudome
        5. HDFS-8827.2.patch
          4 kB
          Jing Zhao
        6. HDFS-8827.1.patch
          2 kB
          Takuya Fukudome

        Activity

          People

            walter.k.su Walter Su
            tfukudom Takuya Fukudome
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: