Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
0.6.2
-
None
-
None
Description
getMapOutput does not reliabily detect errors in transmission, which usually ends up with an EOFException in the reduce's sort:
java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:178)
at java.io.DataInputStream.readFully(DataInputStream.java:152)
at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:952)
at org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:937)
at org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:928)
at org.apache.hadoop.io.SequenceFile$Sorter$SortPass.run(SequenceFile.java:1594)
at org.apache.hadoop.io.SequenceFile$Sorter.sortPass(SequenceFile.java:1523)
at org.apache.hadoop.io.SequenceFile$Sorter.sort(SequenceFile.java:1496)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:240)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1173)