[HADOOP-9150] Unnecessary DNS resolution attempts for logical URIs

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.0.2-alpha, 3.0.0-alpha1
• Fix Version/s: 2.1.0-beta
• Component/s: fs/s3, ha, performance, viewfs
• Target Version/s: 2.0.3-alpha
• Hadoop Flags: Reviewed
• Assignee: Todd Lipcon
• Reporter: Todd Lipcon
• Created: 17/Dec/12 18:54
• Updated: 12/May/16 18:21
• Resolved: 28/Mar/13 20:59

Description

In the FileSystem code, we accidentally try to DNS-resolve the logical name before it is converted to an actual domain name. In some DNS setups, this can cause a big slowdown - eg in one misconfigured cluster we saw a 2-3x drop in terasort throughput, since every task wasted a lot of time waiting for slow "not found" responses from DNS.

Issue Links

• is related to: HDFS-6789 TestDFSClientFailover.testFileContextDoesntDnsResolveLogicalURI and TestDFSClientFailover.testDoesntDnsResolveLogicalURI failing on jdk7