[MAPREDUCE-1813] NPE in PipeMapred.MRErrorThread

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.20.1
• Fix Version/s: 0.22.0
• Component/s: contrib/streaming
• Hadoop Flags: Reviewed
• Release Note: Fixed an NPE in streaming that occurs when there is no input to reduce and the streaming reducer sends status updates by writing "reporter:status: xxx" statements to stderr.
• Assignee: Ravi Gummadi
• Reporter: Amareshwari Sriramadasu
• Created: 25/May/10 08:07
• Updated: 12/Dec/11 06:19
• Resolved: 11/Jun/10 11:41

Description

Some reduce tasks fail with following NPE
java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:325)
at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:540)
at org.apache.hadoop.streaming.PipeReducer.close(PipeReducer.java:137)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:474)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:412)
at org.apache.hadoop.mapred.Child.main(Child.java:159)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.setStatus(PipeMapRed.java:517)
at org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.run(PipeMapRed.java:449)

Issue Links

• is related to: HADOOP-3089 streaming should accept stderr from task before first key arrives
• is related to: HADOOP-4620 Streaming mapper never completes if the mapper does not write to stdout