[HDFS-2737] HA: Automatically trigger log rolls periodically on the active NN

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: HA branch (HDFS-1623)
• Fix Version/s: HA branch (HDFS-1623)
• Component/s: ha, namenode
• Target Version/s: HA branch (HDFS-1623)
• Hadoop Flags: Reviewed
• Assignee: Aaron Myers
• Reporter: Todd Lipcon
• Created: 30/Dec/11 22:27
• Updated: 02/May/13 02:29
• Resolved: 21/Jan/12 03:18

Description

Currently, the edit log tailing process can only read finalized log segments. So, if the active NN is not rolling its logs periodically, the SBN will lag a lot. This also causes many datanode messages to be queued up in the PendingDatanodeMessage structure.

To combat this, the active NN needs to roll its logs periodically – perhaps based on a time threshold, or perhaps based on a number of transactions. I'm not sure yet whether it's better to have the NN roll on its own or to have the SBN ask the active NN to roll its logs.

Issue Links

• is depended upon by: HDFS-2823 HA: Transition to active can cause NN deadlock