Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1813

NPE in PipeMapred.MRErrorThread

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.20.1
    • 0.22.0
    • contrib/streaming
    • None
    • Reviewed
    • Fixed an NPE in streaming that occurs when there is no input to reduce and the streaming reducer sends status updates by writing "reporter:status: xxx" statements to stderr.

    Description

      Some reduce tasks fail with following NPE
      java.lang.RuntimeException: java.lang.NullPointerException
      at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:325)
      at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:540)
      at org.apache.hadoop.streaming.PipeReducer.close(PipeReducer.java:137)
      at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:474)
      at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:412)
      at org.apache.hadoop.mapred.Child.main(Child.java:159)
      Caused by: java.lang.NullPointerException
      at org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.setStatus(PipeMapRed.java:517)
      at org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.run(PipeMapRed.java:449)

      Attachments

        1. 1813.patch
          7 kB
          Ravi Gummadi
        2. 1813.v1.2.patch
          13 kB
          Ravi Gummadi
        3. 1813.v1.3.patch
          31 kB
          Ravi Gummadi
        4. 1813.v1.4.patch
          32 kB
          Ravi Gummadi
        5. 1813.v1.patch
          12 kB
          Ravi Gummadi

        Issue Links

          Activity

            People

              ravidotg Ravi Gummadi
              amareshwari Amareshwari Sriramadasu
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: