[HADOOP-11890] Uber-JIRA: Hadoop should support IPv6

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: net
• Labels: ipv6
• Assignee: Nate Edel
• Reporter: Nate Edel
• Created: 29/Apr/15 17:46
• Updated: 16/Jul/21 13:55

Description

Hadoop currently treats IPv6 as unsupported. Track related smaller issues to support IPv6.

(Current case here is mainly HBase on HDFS, so any suggestions about other test cases/workload are really appreciated.)

Issue Links

• is cloned by: HADOOP-17800 CLONE - Uber-JIRA: Hadoop should support IPv6
• is depended upon by: HADOOP-17543 HDFS Put was failed with IPV6 cluster
• is related to: HDFS-14762 "Path(Path/String parent, String child)" will fail when "child" contains ":"
• is related to: YARN-1226 Inconsistent hostname leads to low data locality on IPv6 hosts
• is related to: YARN-8604 Modify WebApps.Builder<T>#at to parse IPv6 address
• is related to: HADOOP-11574 Uber-JIRA: improve Hadoop network resilience & diagnostics
• is related to: MAPREDUCE-6519 Avoid unsafe split and append on fields that might be IPv6 literals
• is related to: MAPREDUCE-6523 hadoop-yarn Avoid unsafe split and append on fields that might be IPv6 literals
• is related to: YARN-4283 Avoid unsafe split and append on fields that might be IPv6 literals
• relates to: HDFS-9266 Avoid unsafe split and append on fields that might be IPv6 literals
• links to: https://github.com/ghelmling/hadoop/tree/branch-2.7.3-ipv6

Sub-Tasks

• Fix HDFS client gets errors trying to to connect to IPv6 DataNode: Resolved
• Allow hadoop.sh to bind to ipv6 conditionally: Resolved
• DNS.getHosts triggers an ArrayIndexOutOfBoundsException in reverseDNS if one of the interfaces is IPv6: Resolved
• org.apache.hadoop.net.TestDNS failing with NumberFormatException -IPv6 related?: Resolved
• DNS#reverseDns fails with a NumberFormatException when using an IPv6 DNS server: Resolved
• better Logging in DNS.java: Open
• NameNode should bind on both IPv6 and IPv4 if running on dual-stack machine and IPv6 enabled: Resolved
• Add support for include/exclude lists on IPv6 setup: Resolved
• Fix Hadoop should avoid unsafe split and append on fields that might be IPv6 literals: Resolved
• Don't prefer IPV4 stack on tests.: Resolved
• Hadoop-common - Avoid unsafe split and append on fields that might be IPv6 literals: Resolved
• Fix TestNetUtils and TestSecurityUtil when localhost is ipv6 only: Resolved
• Fix libexec scripts that unconditionally add on preferIPv4: Resolved
• Clean up IPv6 code: Open
• Update documentation to reflect IPv6 Hadoop cluster setup: Open
• NetUtils.createSocketAddr was not handling IPV6 scoped address: Resolved
• IPV6 support in Netutils#createSocketAddress: Resolved
• HDFS Put was failed with IPV6 cluster: Open
• Support IPV6 with IP for internal and external communication: In Progress
• Support IPv6 for ChecksumFileSystem: Open