Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
In the interest of making coordinated/consistent reads easier to complete with low latency, it is advantageous to reduce the time between when a transaction is applied on the ANN and when it is applied on the SbNN. We propose adding a new "fast path" which can be used to tail edits when low latency is desired. We leave the existing tailing logic in place, and fall back to this path on startup, recovery, and when the fast path encounters unrecoverable errors.
Attachments
Attachments
Issue Links
- depends upon
-
HDFS-13607 [Edit Tail Fast Path Pt 1] Enhance JournalNode with an in-memory cache of recent edit transactions
- Resolved
-
HDFS-13608 [Edit Tail Fast Path Pt 2] Add ability for JournalNode to serve edits via RPC
- Resolved
-
HDFS-13609 [Edit Tail Fast Path Pt 3] NameNode-side changes to support tailing edits via RPC
- Resolved
-
HDFS-13610 [Edit Tail Fast Path Pt 4] Cleanup: integration test, documentation, remove unnecessary dummy sync
- Resolved
- is related to
-
HADOOP-16208 Do Not Log InterruptedException in Client
- Resolved
- relates to
-
HDFS-13789 Reduce logging frequency of QuorumJournalManager#selectInputStreams
- Resolved
-
HDFS-16493 [SBN Read]When fast path tail enabled, standby or observer namenode may read uncommitted data
- Open
-
HDFS-14500 NameNode StartupProgress continues to report edit log segments after the LOADING_EDITS phase is finished
- Resolved
Design document with more details coming soon...