Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14349

Edit log may be rolled more frequently than necessary with multiple Standby nodes

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: ha, hdfs, qjm
    • Labels:
    • Target Version/s:

      Description

      When HDFS-14317 was fixed, we tackled the problem that in a cluster with in-progress edit log tailing enabled, a Standby NameNode may never roll the edit logs, which can eventually cause data loss.

      Unfortunately, in the process, it was made so that if there are multiple Standby NameNodes, they will all roll the edit logs at their specified frequency, so the edit log will be rolled X times more frequently than they should be (where X is the number of Standby NNs). This is not as bad as the original bug since rolling frequently does not affect correctness or data availability, but may degrade performance by creating more edit log segments than necessary.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ekanth Ekanth Sethuramalingam
                Reporter:
                xkrogen Erik Krogen
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated: