Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-1573

Support for 0 reducers in PIPES

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.14.0
    • 0.15.0
    • None
    • None

    Description

      Under pipes, when specifying number of reducers to zero, all map tasks fail:

      java.io.IOException: pipe child exception
      at org.apache.hadoop.mapred.pipes.Application.abort(Application.java:124)
      at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:82)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:189)
      at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1763)
      Caused by: java.net.SocketException: Connection reset
      at java.net.SocketInputStream.read(SocketInputStream.java:168)
      at java.net.SocketInputStream.read(SocketInputStream.java:182)
      at java.io.DataInputStream.readByte(DataInputStream.java:248)
      at org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:313)
      at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:335)
      at org.apache.hadoop.mapred.pipes.BinaryProtocol$UplinkReaderThread.run(BinaryProtocol.java:101)

      Attachments

        1. pipe-no-reduce.patch
          4 kB
          Owen O'Malley

        Activity

          People

            omalley Owen O'Malley
            ckunz Christian Kunz
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: