Details
Description
StandbyNode in HDFS is a replica of the active NameNode. The states of the NameNodes are coordinated via the journal. It is natural to consider StandbyNode as a read-only replica. As with any replicated distributed system the problem of stale reads should be resolved. Our main goal is to provide reads from standby in a consistent way in order to enable a wide range of existing applications running on top of HDFS.
Attachments
Attachments
Issue Links
- breaks
-
HDFS-14435 ObserverReadProxyProvider is unable to properly fetch HAState from Standby NNs
- Resolved
- incorporates
-
HDFS-15751 Add documentation for msync() API to filesystem.md
- Resolved
- is related to
-
HDFS-14245 Class cast error in GetGroups with ObserverReadProxyProvider
- Resolved
-
HDFS-13664 Refactor ConfiguredFailoverProxyProvider to make inheritance easier
- Resolved
-
HDFS-10702 Add a Client API and Proxy Provider to enable stale read from Standby
- Open
-
HDFS-6440 Support more than 2 NameNodes
- Resolved
-
HADOOP-17477 [SBN read] Implement msync() for ViewFS
- Open
-
HDFS-14205 Backport HDFS-6440 to branch-2
- Resolved
-
HDFS-10519 Add a configuration option to enable in-progress edit log tailing
- Resolved
-
HDFS-13735 Make QJM HTTP URL connection timeout configurable
- Resolved
-
HDFS-13814 Remove super user privilege requirement for NameNode.getServiceStatus
- Resolved
- relates to
-
HDFS-14272 [SBN read] ObserverReadProxyProvider should sync with active txnID on startup
- Resolved
-
HDFS-14279 [SBN Read] Race condition in ObserverReadProxyProvider
- Resolved
-
HDFS-14347 Restore a comment line mistakenly removed in ProtobufRpcEngine
- Resolved
-
HDFS-14204 Backport HDFS-12943 to branch-2
- Resolved
-
HDFS-14211 [Consistent Observer Reads] Allow for configurable "always msync" mode
- Resolved
-
HDFS-14250 [SBN read] msync should sync with active NameNode to fetch the latest stateID
- Resolved
-
HDFS-14271 [SBN read] StandbyException is logged if Observer is the first NameNode
- Patch Available
-
HDFS-14573 Backport Standby Read to branch-3
- Resolved
The design document covers motivation, main requirements, and potential solutions. It describes the consistency model, gives examples and use cases, introduces the new API, discusses implementation details. The roadmap lists four major stages and sets HDFS-10702 as the initial stage.