[HADOOP-4620] Streaming mapper never completes if the mapper does not write to stdout

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.17.2
• Fix Version/s: 0.18.3
• Hadoop Flags: Reviewed

Release Note

This patch HADOOP-4620.patch
(1) solves the hanging problem on map side with empty input and nonempty output — this map task generates output properly to intermediate files similar to other map tasks.
(2) solves the problem of hanging reducer with empty input to reduce task and nonempty output — this reduce task doesn't generate output if input to reduce task is empty.

• Assignee: Ravi Gummadi
• Reporter: Runping Qi
• Created: 08/Nov/08 00:52
• Updated: 08/Jun/10 08:59
• Resolved: 12/Dec/08 05:32

Description

A mapper of a streaming job has empty input data and thus it produces no output.
The task never completes.

The following are the last two lines from the task log:
2008-11-07 21:59:48,254 INFO org.apache.hadoop.streaming.PipeMapRed: PipeMapRed exec [/usr/bin/perl, xxx]
2008-11-07 21:59:48,330 INFO org.apache.hadoop.streaming.PipeMapRed: mapRedFinished

Issue Links

• relates to: MAPREDUCE-1813 NPE in PipeMapred.MRErrorThread