[HDFS-13522] HDFS-13522: Add federated nameservices states to client protocol and propagate it between routers and clients.

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.4.0
• Component/s: federation, namenode
• Labels: pull-request-available
• Assignee: Simbarashe Dzinamarira Assign to me
• Reporter: Erik Krogen
• Created: 03/May/18 00:05
• Updated: 12/Sep/23 15:19
• Resolved: 09/Sep/22 22:52

Description

Changes will need to occur to the router to support the new observer node.

One such change will be to make the router understand the observer state, e.g. FederationNamenodeServiceState.

This patch captures the state of all namespaces in the routers and propagates it to clients. A follow up patch will change router behavior to direct requests to the observer.

 

 

Issue Links

• causes: HDFS-16826 [RBF SBN] ConnectionManager should advance the client stateId for every request
• causes: HDFS-16831 [RBF SBN] GetNamenodesForNameserviceId should shuffle Observer NameNodes every time
• causes: HDFS-16837 [RBF SBN] ClientGSIContext should merge RouterFederatedStates to get the max state id for each namespace
• Dependent: HDFS-16589 RBF: Support cluster level SBN read switch in Router-Based Federation
• incorporates: HDFS-14887 RBF: In Router Web UI, Observer Namenode Information displaying as Unavailable
• is related to: HDFS-13286 Add haadmin commands to transition between standby and observer
• is related to: HDFS-16826 [RBF SBN] ConnectionManager should advance the client stateId for every request
• is related to: HDFS-16831 [RBF SBN] GetNamenodesForNameserviceId should shuffle Observer NameNodes every time
• is related to: HDFS-16834 PoolAlignmentContext should not max poolLocalStateId with sharedGlobalStateId when sending requests to the namenode.
• is related to: HDFS-16837 [RBF SBN] ClientGSIContext should merge RouterFederatedStates to get the max state id for each namespace
• is related to: HDFS-16827 [RBF SBN] RouterStateIdContext shouldn't update the ResponseState if client doesn't use ObserverReadProxyProvider
• is related to: HDFS-16889 Backport JIRAs related to RBF SBN to branch-3.3
• relates to: HDFS-16821 Fix regression in HDFS-13522 that enables observer reads by default.
• requires: HADOOP-18345 Enhance client protocol to propagate last seen state IDs for multiple nameservices.
• requires: HADOOP-18406 Adds alignment context to call path for creating RPC proxy with multiple connections per user.
• split to: HDFS-16767 RBF: Support observer node from Router-Based Federation
• links to: GitHub Pull Request #3005
• links to: GitHub Pull Request #4127
• links to: GitHub Pull Request #4311
• links to: GitHub Pull Request #4441
• links to: GitHub Pull Request #4523
• links to: GitHub Pull Request #4883