[HDFS-3092] Enable journal protocol based editlog streaming for standby namenode

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 0.23.3, 2.0.0-alpha
• Component/s: ha, namenode
• Assignee: Suresh Srinivas
• Reporter: Suresh Srinivas
• Created: 14/Mar/12 13:44
• Updated: 10/Mar/15 04:36
• Resolved: 08/Jul/13 18:40

Description

Currently standby namenode relies on reading shared editlogs to stay current with the active namenode, for namespace changes. BackupNode used streaming edits from active namenode for doing the same. This jira is to explore using journal protocol based editlog streams for the standby namenode. A daemon in standby will get the editlogs from the active and write it to local edits. To begin with, the existing standby mechanism of reading from a file, will continue to be used, instead of from shared edits, from the local edits.

Issue Links

• is depended upon by: HDFS-3278 Umbrella Jira for HDFS-HA Phase 2
• is related to: HDFS-3077 Quorum-based protocol for reading and writing edit logs
• relates to: HDFS-3204 Minor modification to JournalProtocol.proto to make it generic

Sub-Tasks

• Add JournalManager implementation to JournalDaemons for storing edits: Open
• Persist the epoch received by the JournalService: Open
• Handle block pool ID in journal service: Open
• handle creation time also in journal service: Open
• stream the edit segments to NameNode when NameNode starts up: Open
• remove JournalServiceProtocols: Open
• Enable standby namenode to tail edits from journal service: Open
• Replaced Kerberized SSL for journal segment transfer with SPNEGO-based solution: Open