Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-15421

IBR leak causes standby NN to be stuck in safe mode

    XMLWordPrintableJSON

Details

    Description

      After HDFS-14941, update of the global gen stamp is delayed in certain situations.  This makes the last set of incremental block reports from append "from future", which causes it to be simply re-queued to the pending DN message queue, rather than processed to complete the block.  The last set of IBRs will leak and never cleaned until it transitions to active.  The size of pendingDNMessages constantly grows until then.

      If a leak happens while in a startup safe mode, the namenode will never be able to come out of safe mode on its own.

      Attachments

        1. HDFS-15421.002.patch
          13 kB
          Akira Ajisaka
        2. HDFS-15421.003.patch
          8 kB
          Akira Ajisaka
        3. HDFS-15421.004.patch
          9 kB
          Akira Ajisaka
        4. HDFS-15421.005.patch
          11 kB
          Akira Ajisaka
        5. HDFS-15421.006.patch
          11 kB
          Akira Ajisaka
        6. HDFS-15421.007.patch
          12 kB
          Akira Ajisaka
        7. HDFS-15421-000.patch
          12 kB
          Akira Ajisaka
        8. HDFS-15421-001.patch
          13 kB
          Akira Ajisaka

        Issue Links

          Activity

            People

              aajisaka Akira Ajisaka
              kihwal Kihwal Lee
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: