Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1813

NPE in PipeMapred.MRErrorThread

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.20.1
    • 0.22.0
    • contrib/streaming
    • None
    • Reviewed
    • Fixed an NPE in streaming that occurs when there is no input to reduce and the streaming reducer sends status updates by writing "reporter:status: xxx" statements to stderr.

    Description

      Some reduce tasks fail with following NPE
      java.lang.RuntimeException: java.lang.NullPointerException
      at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:325)
      at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:540)
      at org.apache.hadoop.streaming.PipeReducer.close(PipeReducer.java:137)
      at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:474)
      at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:412)
      at org.apache.hadoop.mapred.Child.main(Child.java:159)
      Caused by: java.lang.NullPointerException
      at org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.setStatus(PipeMapRed.java:517)
      at org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.run(PipeMapRed.java:449)

      Attachments

        1. 1813.patch
          7 kB
          Ravi Gummadi
        2. 1813.v1.patch
          12 kB
          Ravi Gummadi
        3. 1813.v1.2.patch
          13 kB
          Ravi Gummadi
        4. 1813.v1.3.patch
          31 kB
          Ravi Gummadi
        5. 1813.v1.4.patch
          32 kB
          Ravi Gummadi

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ravidotg Ravi Gummadi
            amareshwari Amareshwari Sriramadasu
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment