[HDFS-11306] Print remaining edit logs from buffer if edit log can't be rolled.

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.9.0, 3.0.0-alpha2
• Component/s: ha, namenode
• Target Version/s: 2.9.0, 3.0.0-beta1
• Hadoop Flags: Reviewed
• Assignee: Wei-Chiu Chuang
• Reporter: Wei-Chiu Chuang
• Created: 09/Jan/17 21:36
• Updated: 02/Oct/19 17:16
• Resolved: 17/Jan/17 21:15

Description

In HDFS-10943 yzhangal reported that edit log can not be rolled due to unexpected edit logs lingering in the buffer.

Unable to root cause the bug, I propose that we dump the remaining edit logs in the buffer into namenode log, before crashing namenode. Use this new capability to find the ops that sneaks into the buffer unexpectedly, and hopefully catch the bug.

This effort is orthogonal, but related to HDFS-11292, which adds additional informational logs to help debug this issue.

Issue Links

• relates to: HDFS-10943 rollEditLog expects empty EditsDoubleBuffer.bufCurrent which is not guaranteed
• relates to: HDFS-11292 log lastWrittenTxId etc info in logSyncAll