Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-15421

IBR leak causes standby NN to be stuck in safe mode

    XMLWordPrintableJSON

    Details

      Description

      After HDFS-14941, update of the global gen stamp is delayed in certain situations.  This makes the last set of incremental block reports from append "from future", which causes it to be simply re-queued to the pending DN message queue, rather than processed to complete the block.  The last set of IBRs will leak and never cleaned until it transitions to active.  The size of pendingDNMessages constantly grows until then.

      If a leak happens while in a startup safe mode, the namenode will never be able to come out of safe mode on its own.

        Attachments

        1. HDFS-15421-000.patch
          12 kB
          Akira Ajisaka
        2. HDFS-15421-001.patch
          13 kB
          Akira Ajisaka
        3. HDFS-15421.002.patch
          13 kB
          Akira Ajisaka
        4. HDFS-15421.003.patch
          8 kB
          Akira Ajisaka
        5. HDFS-15421.004.patch
          9 kB
          Akira Ajisaka
        6. HDFS-15421.005.patch
          11 kB
          Akira Ajisaka
        7. HDFS-15421.006.patch
          11 kB
          Akira Ajisaka
        8. HDFS-15421.007.patch
          12 kB
          Akira Ajisaka

          Issue Links

            Activity

              People

              • Assignee:
                aajisaka Akira Ajisaka
                Reporter:
                kihwal Kihwal Lee
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: