[HDFS-2683] Authority-based lookup of proxy provider fails if path becomes canonicalized

• Type: Sub-task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: HA branch (HDFS-1623)
• Fix Version/s: HA branch (HDFS-1623)
• Component/s: ha, hdfs-client
• Target Version/s: HA branch (HDFS-1623)
• Hadoop Flags: Reviewed
• Assignee: Todd Lipcon
• Reporter: Todd Lipcon
• Created: 14/Dec/11 23:40
• Updated: 15/Dec/11 00:43
• Resolved: 15/Dec/11 00:43

Description

When testing MapReduce on top of an HA cluster we ran into the following bug: some uses of HDFS paths go through a canonicalization step which ensures that the authority component in the URI includes a port number. So our hdfs://logical-nn-uri/foo path turned into hdfs://logical-nn-uri:8020/foo. The code which looks up the failover proxy provider then failed to find the associated config. We should only compare the hostname portion of the URI when looking up proxy providers.