[HDFS-4298] StorageRetentionManager spews warnings when used with QJM

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.3-alpha, 3.0.0-alpha1
• Fix Version/s: 2.1.0-beta
• Component/s: namenode
• Target Version/s: 2.1.0-beta
• Hadoop Flags: Reviewed
• Assignee: Aaron Myers
• Reporter: Todd Lipcon
• Created: 11/Dec/12 00:37
• Updated: 12/May/16 18:16
• Resolved: 22/May/13 19:43

Description

When the NN is configured with a QJM, we see the following warning message every time a checkpoint is made or uploaded:
12/12/10 16:07:52 WARN namenode.FSEditLog: Unable to determine input streams from QJM to [127.0.0.1:13001, 127.0.0.1:13002, 127.0.0.1:13003]. Skipping.
org.apache.hadoop.hdfs.qjournal.client.QuorumException: Got too many exceptions to achieve quorum size 2/3. 3 exceptions thrown:
127.0.0.1:13002: Asked for firstTxId 114837 which is in the middle of file /tmp/jn-2/myjournal/current/edits_0000000000000095185-0000000000000114846
...

This is because, since HDFS-2946, the NN calls selectInputStreams to determine the number of log segments and put a cap on the number. This API throws an exception in the case of QJM if the argument falls in the middle of an edit log boundary.

Issue Links

• is duplicated by: HDFS-4617 warning while purging logs with QJM enabled