Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11306

Print remaining edit logs from buffer if edit log can't be rolled.

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: ha, namenode
    • Labels:
      None

      Description

      In HDFS-10943 Yongjun Zhang reported that edit log can not be rolled due to unexpected edit logs lingering in the buffer.

      Unable to root cause the bug, I propose that we dump the remaining edit logs in the buffer into namenode log, before crashing namenode. Use this new capability to find the ops that sneaks into the buffer unexpectedly, and hopefully catch the bug.

      This effort is orthogonal, but related to HDFS-11292, which adds additional informational logs to help debug this issue.

        Attachments

        1. HDFS-11306.003.patch
          6 kB
          Wei-Chiu Chuang
        2. HDFS-11306.002.patch
          6 kB
          Wei-Chiu Chuang
        3. HDFS-11306.001.patch
          4 kB
          Wei-Chiu Chuang

          Issue Links

            Activity

              People

              • Assignee:
                jojochuang Wei-Chiu Chuang
                Reporter:
                jojochuang Wei-Chiu Chuang
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: