Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4739

NN can miscalculate the number of extra edit log segments to retain

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.4-alpha
    • Fix Version/s: 2.1.0-beta
    • Component/s: namenode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The code in NNStorageRetentionManager#purgeOldStorage is intended to place a cap on the number of extra edit log segments retained beyond what is strictly required to replay the FS history since the last fsimage. In fact this code currently places a limit on the total number of extra edit log segments. If the number of required segments is greater than the configured cap, there will be no data loss, but an ugly error will be thrown and the NN will fail to start.

      The fix is simple, and in the meantime a work-around is just to raise the value of dfs.namenode.max.extra.edits.segments.retained and start the NN.

        Attachments

        1. HDFS-4739.patch
          3 kB
          Aaron Myers
        2. HDFS-4739.patch
          3 kB
          Aaron Myers

          Activity

            People

            • Assignee:
              atm Aaron Myers
              Reporter:
              atm Aaron Myers
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: