[HDFS-15785] Datanode to support using DNS to resolve nameservices to IP addresses to get list of namenodes

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.4.0
• Component/s: datanode
• Labels: pull-request-available
• Assignee: Leon Gao
• Reporter: Leon Gao
• Created: 22/Jan/21 01:03
• Updated: 13/Jul/21 22:02
• Resolved: 13/Jul/21 22:00

Description

Currently as HDFS supports observers, multiple-standby and router, the namenode hosts are changing frequently in large deployment, we can consider supporting https://issues.apache.org/jira/browse/HDFS-14118 on datanode to reduce the need to update config frequently on all datanodes. In that case, datanode and clients can use the same set of config as well.

Basically we can resolve the DNS and generate namenode for each IP behind it.

Issue Links

• is a child of: HDFS-15831 Adopt more DNS resolving for HDFS
• is related to: HDFS-14600 DataNodes using DNS NameNode discovery will not send block reports when failover
• relates to: HDFS-14118 Support using DNS to resolve nameservices to IP addresses
• links to: GitHub Pull Request #2639
• links to: GitHub Pull Request #2639