[MAPREDUCE-1522] FileInputFormat may change the file system of an input path

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 0.20.3
• Hadoop Flags: Reviewed
• Assignee: Tsz-wo Sze
• Reporter: Tsz-wo Sze
• Created: 22/Feb/10 22:00
• Updated: 02/May/13 02:29
• Resolved: 05/Mar/10 22:21

Description

org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(Job job, Path path) uses the default FileSystem but not the FileSystem specified in the path.

//org.apache.hadoop.mapreduce.lib.input.FileInputFormat
  public static void addInputPath(Job job, 
                                  Path path) throws IOException {
    Configuration conf = job.getConfiguration();
    FileSystem fs = FileSystem.get(conf);
    path = path.makeQualified(fs); // the original FileSystem is lost.
    ...
  }

There is a similar problem in FileInputFormat.setInputPaths(..).

Issue Links

• is depended upon by: PIG-1378 har url of the form har:///<path> not usable in Pig scripts (har://hdfs-namenode:port/<path> works)
• is related to: HADOOP-6564 HarFileSystem cannot handle har:///