[HADOOP-3166] SpillThread throws ArrayIndexOutOfBoundsException, which is ignored by MapTask

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 0.17.0
• Hadoop Flags: Reviewed
• Assignee: Christopher Douglas
• Reporter: Christopher Douglas
• Created: 03/Apr/08 21:51
• Updated: 21/May/08 20:06
• Resolved: 04/Apr/08 20:46

Description

There is a flaw in the value bounds checking introduced by HADOOP-2919. When the index table determines the end offset of the value, it fails to detect that the next entry falls off the end of the array. This remains undetected by MapTask, which only notes IOExceptions from the spill thread. Both the boundary checking and the exception handling must be fixed.

Issue Links

• is related to: HADOOP-3154 Job successful but dropping records (when disk full)