Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8178

QJM doesn't move aside stale inprogress edits files

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.3.0, 3.2.1, 3.1.3
    • Component/s: qjm
    • Labels:

      Description

      When a QJM crashes, the in-progress edit log file at that time remains in the file system. When the node comes back, it will accept new edit logs and those stale in-progress files are never cleaned up. QJM treats them as regular in-progress edit log files and tries to finalize them, which potentially causes high memory usage. This JIRA aims to move aside those stale edit log files to avoid this scenario.

        Attachments

        1. HDFS-8178.000.patch
          4 kB
          Zhe Zhang
        2. HDFS-8178.002.patch
          3 kB
          Zhe Zhang
        3. HDFS-8178.003.patch
          4 kB
          Zhe Zhang
        4. HDFS-8178.004.patch
          14 kB
          Istvan Fajth
        5. HDFS-8178.005.patch
          16 kB
          Istvan Fajth
        6. HDFS-8178.006.patch
          16 kB
          Istvan Fajth
        7. HDFS-8178.007.patch
          16 kB
          Istvan Fajth
        8. HDFS-8178.008.addendum
          26 kB
          Istvan Fajth
        9. HDFS-8178.008.merged
          37 kB
          Istvan Fajth
        10. HDFS-8178.008.patch
          19 kB
          Istvan Fajth
        11. HDFS-8178.branch-3.2.patch
          19 kB
          Wei-Chiu Chuang
        12. HDFS-8178.branch-3.1.patch
          18 kB
          Istvan Fajth

          Issue Links

            Activity

              People

              • Assignee:
                pifta Istvan Fajth
                Reporter:
                zhz Zhe Zhang
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: