[HADOOP-552] getMapOutput doesn't reliably detect errors and throw to the caller

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.6.2
• Fix Version/s: 0.7.0
• Assignee: Owen O'Malley
• Reporter: Owen O'Malley
• Created: 20/Sep/06 16:26
• Updated: 08/Jul/09 16:51
• Resolved: 02/Oct/06 18:16

Description

getMapOutput does not reliabily detect errors in transmission, which usually ends up with an EOFException in the reduce's sort:

java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:178)
at java.io.DataInputStream.readFully(DataInputStream.java:152)
at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:952)
at org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:937)
at org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:928)
at org.apache.hadoop.io.SequenceFile$Sorter$SortPass.run(SequenceFile.java:1594)
at org.apache.hadoop.io.SequenceFile$Sorter.sortPass(SequenceFile.java:1523)
at org.apache.hadoop.io.SequenceFile$Sorter.sort(SequenceFile.java:1496)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:240)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1173)