[HADOOP-1452] map output transfers of more than 2^31 bytes output are failing

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.13.0
• Fix Version/s: 0.13.0
• Assignee: Owen O'Malley
• Reporter: Christian Kunz
• Created: 01/Jun/07 20:16
• Updated: 08/Jul/09 16:52
• Resolved: 01/Jun/07 21:26

Description

Symptom:

WARN org.apache.hadoop.mapred.ReduceTask: java.io.IOException: Incomplete map output received for http://<host>:50060/mapOutput?map=task_0026_m_000298_0&reduce=61 (2327458761 instead of 2327347307)
WARN org.apache.hadoop.mapred.ReduceTask: task_0026_r_000061_0 adding host <host> to penalty box, next contact in 263 seconds

Besides failing to fetch data, the reduce will retry forever. This should be limited.

Source of the problem:

in mapred/TaskTracker.java the variable totalRead keeping track what is sent to the reducer should be declared as long:

...
int totalRead = 0;
int len = mapOutputIn.read(buffer, 0,
partLength < MAX_BYTES_TO_READ
? (int)partLength : MAX_BYTES_TO_READ);
while (len > 0) {
try

{ outStream.write(buffer, 0, len); outStream.flush(); }

catch (IOException ie)

{ isInputException = false; throw ie; }

totalRead += len;
if (totalRead == partLength) break;
len = mapOutputIn.read(buffer, 0,
(partLength - totalRead) < MAX_BYTES_TO_READ
? (int)(partLength - totalRead) : MAX_BYTES_TO_READ);
}
...