Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4298

StorageRetentionManager spews warnings when used with QJM

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.0.3-alpha, 3.0.0-alpha1
    • 2.1.0-beta
    • namenode
    • None
    • Reviewed

    Description

      When the NN is configured with a QJM, we see the following warning message every time a checkpoint is made or uploaded:
      12/12/10 16:07:52 WARN namenode.FSEditLog: Unable to determine input streams from QJM to [127.0.0.1:13001, 127.0.0.1:13002, 127.0.0.1:13003]. Skipping.
      org.apache.hadoop.hdfs.qjournal.client.QuorumException: Got too many exceptions to achieve quorum size 2/3. 3 exceptions thrown:
      127.0.0.1:13002: Asked for firstTxId 114837 which is in the middle of file /tmp/jn-2/myjournal/current/edits_0000000000000095185-0000000000000114846
      ...

      This is because, since HDFS-2946, the NN calls selectInputStreams to determine the number of log segments and put a cap on the number. This API throws an exception in the case of QJM if the argument falls in the middle of an edit log boundary.

      Attachments

        1. HDFS-4298.patch
          30 kB
          Aaron Myers
        2. HDFS-4298.patch
          29 kB
          Aaron Myers
        3. HDFS-4298.patch
          29 kB
          Aaron Myers

        Issue Links

          Activity

            People

              atm Aaron Myers
              tlipcon Todd Lipcon
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: