Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8178

QJM doesn't move aside stale inprogress edits files

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.3.0, 3.2.1, 3.1.3
    • Component/s: qjm
    • Labels:

      Description

      When a QJM crashes, the in-progress edit log file at that time remains in the file system. When the node comes back, it will accept new edit logs and those stale in-progress files are never cleaned up. QJM treats them as regular in-progress edit log files and tries to finalize them, which potentially causes high memory usage. This JIRA aims to move aside those stale edit log files to avoid this scenario.

        Attachments

        1. HDFS-8178.branch-3.1.patch
          18 kB
          Istvan Fajth
        2. HDFS-8178.branch-3.2.patch
          19 kB
          Wei-Chiu Chuang
        3. HDFS-8178.008.patch
          19 kB
          Istvan Fajth
        4. HDFS-8178.008.merged
          37 kB
          Istvan Fajth
        5. HDFS-8178.008.addendum
          26 kB
          Istvan Fajth
        6. HDFS-8178.007.patch
          16 kB
          Istvan Fajth
        7. HDFS-8178.006.patch
          16 kB
          Istvan Fajth
        8. HDFS-8178.005.patch
          16 kB
          Istvan Fajth
        9. HDFS-8178.004.patch
          14 kB
          Istvan Fajth
        10. HDFS-8178.003.patch
          4 kB
          Zhe Zhang
        11. HDFS-8178.002.patch
          3 kB
          Zhe Zhang
        12. HDFS-8178.000.patch
          4 kB
          Zhe Zhang

          Issue Links

            Activity

              People

              • Assignee:
                pifta Istvan Fajth
                Reporter:
                zhz Zhe Zhang
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: