[HDFS-10421] The nameservice for a Hadoop Namenode HA should be discoverable across clusters

• Type: New Feature
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: ha, hdfs-client
• Assignee: Unassigned
• Reporter: Mahipal Jupalli
• Created: 18/May/16 00:14
• Updated: 18/May/16 04:28

Description

Name service configured in NameNode HA mode, should be discoverable across clusters without having to restart all the clusters every time a new cluster is deployed.

Solution #1: One solution would be to add the nameservice configurations to all the hdfs-site.xml files in the clusters involved.
Cons: We cannot restart all clusters every time we deploy a new cluster.

We are developing a custom class to extend org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider, this custom provider will make a REST call to fetch any additional nameservice configurations and cache them locally (expires on a configured time limit). This will fix the issue to resolve the name service across clusters and also eliminate the need to restart all the clusters whenever a new cluster is configured.

Please provide your inputs. Wondering if this would be a good feature to add and if so please assign this to me. I can contribute to this issue.