[HADOOP-3162] Map/reduce stops working with comma separated input paths

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.17.0
• Fix Version/s: 0.17.0

Release Note

The public methods org.apache.hadoop.mapred.JobConf.setInputPath(Path) and org.apache.hadoop.mapred.JobConf.addInputPath(Path) are deprecated. And the methods have the semantics of branch 0.16.
The following public APIs are added in org.apache.hadoop.mapred.FileInputFormat :
public static void setInputPaths(JobConf job, Path... paths);
public static void setInputPaths(JobConf job, String commaSeparatedPaths);
public static void addInputPath(JobConf job, Path path);
public static void addInputPaths(JobConf job, String commaSeparatedPaths);
Earlier code calling JobConf.setInputPath(Path), JobConf.addInputPath(Path) should now call FileInputFormat.setInputPaths(JobConf, Path...) and FileInputFormat.addInputPath(Path) respectively

• Assignee: Amareshwari Sriramadasu
• Reporter: Runping Qi
• Created: 03/Apr/08 17:17
• Updated: 08/Jul/09 16:52
• Resolved: 16/Apr/08 17:36

Description

When a job is given a comma separated input file list, FileInputFormat class throws an exception, complaining the input is invalid:

org.apache.hadoop.mapred.InvalidInputException: Input path doesnt exist : hdfs:/
namenode:port/gridmix/data/MonsterQueryBlockCompressed/part-
00000,/gridmix/data/MonsterQueryBlockCompressed/part-00001,/gridmix/data/Monster
QueryBlockCompressed/part-00002
at org.apache.hadoop.mapred.FileInputFormat.validateInput(FileInputForma
t.java:213)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:705)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:973)
at org.apache.hadoop.mapred.GenericMRLoadGenerator.run(GenericMRLoadGene
rator.java:189)