Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4867

metaSave NPEs when there are invalid blocks in repl queue.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.23.7, 2.0.4-alpha, 0.23.8
    • 2.1.0-beta, 0.23.9
    • namenode
    • None

    Description

      Since metaSave cannot get the inode holding a orphaned/invalid block, it NPEs and stops generating further report. Normally ReplicationMonitor removes them quickly, but if the queue is huge, it takes very long time. Also in safe mode, they stay.

      Attachments

        1. HDFS-4867.branch-0.23.patch
          3 kB
          Konstantin Shvachko
        2. HDFS-4867.branch-0.23.patch
          2 kB
          Plamen Jeliazkov
        3. HDFS-4867.branch-0.23.patch
          6 kB
          Ravi Prakash
        4. HDFS-4867.branch-0.23.patch
          5 kB
          Ravi Prakash
        5. HDFS-4867.branch2.patch
          6 kB
          Ravi Prakash
        6. HDFS-4867.branch2.patch
          4 kB
          Plamen Jeliazkov
        7. HDFS-4867.branch2.patch
          4 kB
          Plamen Jeliazkov
        8. HDFS-4867.branch-2.patch
          1.0 kB
          Plamen Jeliazkov
        9. HDFS-4867.trunk.patch
          2 kB
          Plamen Jeliazkov
        10. HDFS-4867.trunk.patch
          6 kB
          Ravi Prakash
        11. HDFS-4867.trunk.patch
          5 kB
          Plamen Jeliazkov
        12. HDFS-4867.trunk.patch
          4 kB
          Plamen Jeliazkov
        13. testMetaSave.log
          6 kB
          Konstantin Shvachko

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. HDFS-3974 dfsadmin -metasave throws NPE when under-replicated blocks are recently deleted

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-4832 Namenode doesn't change the number of missing blocks in safemode when DNs rejoin or leave

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HDFS-4878 On Remove Block, Block is not Removed from neededReplications queue

          • Major - Major loss of function.
          • Closed

          Activity

            People

              zero45 Plamen Jeliazkov
              kihwal Kihwal Lee
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: