Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4298

StorageRetentionManager spews warnings when used with QJM

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.3-alpha, 3.0.0-alpha1
    • Fix Version/s: 2.1.0-beta
    • Component/s: namenode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      When the NN is configured with a QJM, we see the following warning message every time a checkpoint is made or uploaded:
      12/12/10 16:07:52 WARN namenode.FSEditLog: Unable to determine input streams from QJM to [127.0.0.1:13001, 127.0.0.1:13002, 127.0.0.1:13003]. Skipping.
      org.apache.hadoop.hdfs.qjournal.client.QuorumException: Got too many exceptions to achieve quorum size 2/3. 3 exceptions thrown:
      127.0.0.1:13002: Asked for firstTxId 114837 which is in the middle of file /tmp/jn-2/myjournal/current/edits_0000000000000095185-0000000000000114846
      ...

      This is because, since HDFS-2946, the NN calls selectInputStreams to determine the number of log segments and put a cap on the number. This API throws an exception in the case of QJM if the argument falls in the middle of an edit log boundary.

        Attachments

        1. HDFS-4298.patch
          29 kB
          Aaron Myers
        2. HDFS-4298.patch
          29 kB
          Aaron Myers
        3. HDFS-4298.patch
          30 kB
          Aaron Myers

          Issue Links

            Activity

              People

              • Assignee:
                atm Aaron Myers
                Reporter:
                tlipcon Todd Lipcon
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: