[MAPREDUCE-6121] JobResourceUpdater#compareFs() doesn't handle HA namespaces

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.5.0
• Fix Version/s: 2.8.0, 3.0.0-alpha1
• Component/s: mrv2
• Hadoop Flags: Reviewed
• Assignee: Ray Chiang
• Reporter: Thomas Graves
• Created: 06/Oct/14 14:10
• Updated: 30/Aug/16 01:19
• Resolved: 30/Jun/15 23:54

Description

Looking at the JobSubmitter.compareFs it doesn't look like it properly handles HA namespaces. The code tries to lookup the hostname using InetAddress.getByName, but if you are using namespaces this is going to fail and its going to copy the file when it doesn't need to.

Edit: JobSubmitter was updated to JobResourceUpdater in MAPREDUCE-6267.

Issue Links

• is blocked by: HADOOP-12159 Move DistCpUtils#compareFs() to org.apache.hadoop.fs.FileUtil and fix for HA namespaces