[MAPREDUCE-4888] NLineInputFormat drops data in 1.1 and beyond

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.1.0
• Fix Version/s: 1.1.2
• Component/s: mrv1
• Assignee: Vinod Kumar Vavilapalli
• Reporter: Robert Joseph Evans
• Created: 18/Dec/12 16:50
• Updated: 06/Mar/13 09:55
• Resolved: 21/Dec/12 07:42

Description

When trying to root cause why MAPREDUCE-4782 did not cause us issues on 1.0.2, I found out that HADOOP-7823 introduced essentially the exact same error into org.apache.hadoop.mapred.lib.NLineInputFormat.

In 1.X org.apache.hadoop.mapred.lib.NLineInputFormat and org.apache.hadoop.mapreduce.lib.input.NLineInputFormat are separate implementations. The latter had an off by one error in it until MAPREDUCE-4782 fixed it. The former had no error in it until HADOOP-7823 introduced it in 1.1 and MAPREDUCE-375 combined the implementations together but picked the implementation with the off by one error in 0.21.

I will attach a patch that exposes the error.

Issue Links

• is broken by: HADOOP-7823 port HADOOP-4012 to branch-1 (splitting support for bzip2)