Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4867

metaSave NPEs when there are invalid blocks in repl queue.

VotersStop watchingWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.23.7, 2.0.4-alpha, 0.23.8
    • 2.1.0-beta, 0.23.9
    • namenode
    • None

    Description

      Since metaSave cannot get the inode holding a orphaned/invalid block, it NPEs and stops generating further report. Normally ReplicationMonitor removes them quickly, but if the queue is huge, it takes very long time. Also in safe mode, they stay.

      Attachments

        1. HDFS-4867.trunk.patch
          4 kB
          Plamen Jeliazkov
        2. HDFS-4867.trunk.patch
          5 kB
          Plamen Jeliazkov
        3. HDFS-4867.branch-0.23.patch
          5 kB
          Ravi Prakash
        4. HDFS-4867.branch2.patch
          4 kB
          Plamen Jeliazkov
        5. HDFS-4867.branch2.patch
          4 kB
          Plamen Jeliazkov
        6. testMetaSave.log
          6 kB
          Konstantin Shvachko
        7. HDFS-4867.branch-0.23.patch
          6 kB
          Ravi Prakash
        8. HDFS-4867.branch2.patch
          6 kB
          Ravi Prakash
        9. HDFS-4867.trunk.patch
          6 kB
          Ravi Prakash
        10. HDFS-4867.branch-0.23.patch
          2 kB
          Plamen Jeliazkov
        11. HDFS-4867.branch-2.patch
          1.0 kB
          Plamen Jeliazkov
        12. HDFS-4867.trunk.patch
          2 kB
          Plamen Jeliazkov
        13. HDFS-4867.branch-0.23.patch
          3 kB
          Konstantin Shvachko

        Issue Links

        is duplicated by

        Bug - A problem which impairs or prevents the functions of the product. HDFS-3974 dfsadmin -metasave throws NPE when under-replicated blocks are recently deleted

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Open
        relates to

        Bug - A problem which impairs or prevents the functions of the product. HDFS-4832 Namenode doesn't change the number of missing blocks in safemode when DNs rejoin or leave

        • Critical - Crashes, loss of data, severe memory leak.
        • Closed

        Bug - A problem which impairs or prevents the functions of the product. HDFS-4878 On Remove Block, Block is not Removed from neededReplications queue

        • Major - Major loss of function.
        • Closed

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            zero45 Plamen Jeliazkov
            kihwal Kihwal Lee
            Votes:
            0 Vote for this issue
            Watchers:
            8 Stop watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment