[MAPREDUCE-3952] In MR2, when Total input paths to process == 1, CombinefileInputFormat.getSplits() returns 0 split.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.23.3, 2.0.0-alpha
• Fix Version/s: 2.0.0-alpha, 0.23.7
• Component/s: mrv2
• Hadoop Flags: Reviewed
• Assignee: Bhallamudi Venkata Siva Kamesh
• Reporter: Zhenxiao Luo
• Created: 01/Mar/12 05:10
• Updated: 10/Mar/15 04:33
• Resolved: 06/Mar/12 00:00

Description

Hive get unexpected result when using MR2(When using MR1, always get expected result).

In MR2, when Total input paths to process == 1, CombinefileInputFormat.getSplits() returns 0 split.

The calling code in Hive, in Hadoop23Shims.java:

InputSplit[] splits = super.getSplits(job, numSplits);

this get splits.length == 0.

In MR1, everything goes fine, the calling code in Hive, in Hadoop20Shims.java:

CombineFileSplit[] splits = (CombineFileSplit[]) super.getSplits(job, numSplits);

this get splits.length == 1.