[HADOOP-1090] In SortValidator, the check for whether a file belongs to sort-input or sort-output dir is weak

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.12.2
• Fix Version/s: 0.13.0
• Assignee: Arun Murthy
• Reporter: Devaraj Das
• Created: 08/Mar/07 13:02
• Updated: 08/Jul/09 16:52
• Resolved: 01/Apr/07 19:39

Description

In SortValidator, Maps invoke the method called deduceInputFile in the configure method. The deduceInputFile is supposed to return whether the input file belongs to the sort-input directory or the sort-output directory. However, the check that deduceInputFile does - inputFile.toString().startsWith(inputPaths[0].toString()) - is not totally correct. The check will always returns true for inputPaths like /user/foo/smallInput/<filenames>, /user/foo/smallInput-sorted/<filenames>. This finally causes the SortValidator to declare the sort output as incorrect.