[HDFS-13610] [Edit Tail Fast Path Pt 4] Cleanup: integration test, documentation, remove unnecessary dummy sync

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: HDFS-12943, 3.3.0
• Component/s: ha, journal-node, namenode
• Target Version/s: HDFS-12943
• Hadoop Flags: Reviewed
• Assignee: Erik Krogen
• Reporter: Erik Krogen
• Created: 23/May/18 16:12
• Updated: 24/Dec/18 18:27
• Resolved: 17/Jul/18 18:36

Description

See HDFS-13150 for full design.

This JIRA is targeted at cleanup tasks:

• Add in integration testing. We can expand TestStandbyInProgressTail
• Documentation in HDFSHighAvailabilityWithQJM
• Remove the dummy sync added as part of HDFS-10519; it is unnecessary since now in-progress tailing does not rely on the JN committedTxnId

A few bugs are also fixed:

• Due to changes in HDFS-13609 to enable use of the RPC mechanism whenever inProgressOK is true, there were codepaths which would use the RPC mechanism even when dfs.ha.tail-edits.in-progress was false, meaning that the JNs did not enable the cache. Update the QJM logic to only use selectRpcInputStreams if this config is true.
• Fix a false error logged when the layout version changes
• Fix the logging when a layout version change occurs to avoid printing out a placeholder value (Integer.MAX_VALUE)

Issue Links

• depends upon: HDFS-13609 [Edit Tail Fast Path Pt 3] NameNode-side changes to support tailing edits via RPC
• is depended upon by: HDFS-13150 [Edit Tail Fast Path] Allow SbNN to tail in-progress edits from JN via RPC