[MAPREDUCE-5008] Merger progress miscounts with respect to EOF_MARKER

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.3-alpha
• Fix Version/s: 1.2.0, 2.1.0-beta
• Hadoop Flags: Reviewed
• Assignee: Sandy Ryza
• Reporter: Sandy Ryza
• Created: 15/Feb/13 19:21
• Updated: 15/May/13 05:15
• Resolved: 27/Feb/13 10:43

Description

After MAPREDUCE-2264, a segment's raw data length is calculated without the EOF_MARKER bytes. However, when the merge is counting how many bytes it processed, it includes the marker. This can cause the merge progress to go above 100%.

Whether these EOF_MARKER bytes should count should be consistent between the two.

This a JIRA instead of an amendment because MAPREDUCE-2264 already went into 2.0.3.