Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4867

metaSave NPEs when there are invalid blocks in repl queue.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.23.7, 2.0.4-alpha, 0.23.8
    • 2.1.0-beta, 0.23.9
    • namenode
    • None

    Description

      Since metaSave cannot get the inode holding a orphaned/invalid block, it NPEs and stops generating further report. Normally ReplicationMonitor removes them quickly, but if the queue is huge, it takes very long time. Also in safe mode, they stay.

      Attachments

        1. HDFS-4867.branch-0.23.patch
          3 kB
          Konstantin Shvachko
        2. HDFS-4867.trunk.patch
          2 kB
          Plamen Jeliazkov
        3. HDFS-4867.branch-2.patch
          1.0 kB
          Plamen Jeliazkov
        4. HDFS-4867.branch-0.23.patch
          2 kB
          Plamen Jeliazkov
        5. HDFS-4867.trunk.patch
          6 kB
          Ravi Prakash
        6. HDFS-4867.branch2.patch
          6 kB
          Ravi Prakash
        7. HDFS-4867.branch-0.23.patch
          6 kB
          Ravi Prakash
        8. testMetaSave.log
          6 kB
          Konstantin Shvachko
        9. HDFS-4867.branch2.patch
          4 kB
          Plamen Jeliazkov
        10. HDFS-4867.branch2.patch
          4 kB
          Plamen Jeliazkov
        11. HDFS-4867.branch-0.23.patch
          5 kB
          Ravi Prakash
        12. HDFS-4867.trunk.patch
          5 kB
          Plamen Jeliazkov
        13. HDFS-4867.trunk.patch
          4 kB
          Plamen Jeliazkov

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. HDFS-3974 dfsadmin -metasave throws NPE when under-replicated blocks are recently deleted

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-4832 Namenode doesn't change the number of missing blocks in safemode when DNs rejoin or leave

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HDFS-4878 On Remove Block, Block is not Removed from neededReplications queue

          • Major - Major loss of function.
          • Closed

          Activity

            People

              zero45 Plamen Jeliazkov
              kihwal Kihwal Lee
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: