Details
-
Bug
-
Status: Closed
-
Blocker
-
Resolution: Fixed
-
0.18.0
-
None
-
None
Description
A couple of reduce tasks failed at IFile.Reader.next, one with:
java.lang.NegativeArraySizeException at org.apache.hadoop.mapred.IFile$Reader.readNextBlock(IFile.java:246) at org.apache.hadoop.mapred.IFile$Reader.next(IFile.java:298) at org.apache.hadoop.mapred.Merger$Segment.next(Merger.java:134) at org.apache.hadoop.mapred.Merger$MergeQueue.adjustPriorityQueue(Merger.java:225) at org.apache.hadoop.mapred.Merger$MergeQueue.next(Merger.java:242) at org.apache.hadoop.mapred.Task$ValuesIterator.readNextKey(Task.java:720) at org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:679) at org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.next(ReduceTask.java:225) at org.apache.hadoop.mapred.lib.IdentityReducer.reduce(IdentityReducer.java:39) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:316) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2157)
On a related note, another failed at:
IFile.java:380
// Position for the next record long skipped = dataIn.skip(recordLength); if (skipped != recordLength) { throw new IOException("Failed to skip past record of length: " + recordLength); }
where recordLength was -17.