Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1888

Streaming overrides user given output key and value types.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.22.0
    • Component/s: contrib/streaming
    • Labels:
      None

      Description

      The following code in StreamJob.java overrides user given output key and value types.

          idResolver.resolve(conf.get(StreamJobConfig.MAP_OUTPUT,
              IdentifierResolver.TEXT_ID));
          conf.setClass(StreamJobConfig.MAP_OUTPUT_READER_CLASS,
            idResolver.getOutputReaderClass(), OutputReader.class);
          job.setMapOutputKeyClass(idResolver.getOutputKeyClass());
          job.setMapOutputValueClass(idResolver.getOutputValueClass());
          
          idResolver.resolve(conf.get(StreamJobConfig.REDUCE_OUTPUT,
              IdentifierResolver.TEXT_ID));
          conf.setClass(StreamJobConfig.REDUCE_OUTPUT_READER_CLASS,
            idResolver.getOutputReaderClass(), OutputReader.class);
          job.setOutputKeyClass(idResolver.getOutputKeyClass());
          job.setOutputValueClass(idResolver.getOutputValueClass());
      
      1. 1888.patch
        9 kB
        Ravi Gummadi
      2. 1888.v1.patch
        25 kB
        Ravi Gummadi
      3. 1888.v2.patch
        26 kB
        Ravi Gummadi
      4. 1888.v3.patch
        29 kB
        Ravi Gummadi
      5. 1888.v4.patch
        34 kB
        Ravi Gummadi

        Issue Links

          Activity

          Amareshwari Sriramadasu created issue -
          Ravi Gummadi made changes -
          Field Original Value New Value
          Assignee Ravi Gummadi [ ravidotg ]
          Ravi Gummadi made changes -
          Attachment 1888.patch [ 12448034 ]
          Ravi Gummadi made changes -
          Attachment 1888.v1.patch [ 12448197 ]
          Ravi Gummadi made changes -
          Attachment 1888.v2.patch [ 12448202 ]
          Amareshwari Sriramadasu made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Amareshwari Sriramadasu made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Ravi Gummadi made changes -
          Attachment 1888.v3.patch [ 12448337 ]
          Ravi Gummadi made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Amareshwari Sriramadasu made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Amareshwari Sriramadasu made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Ravi Gummadi made changes -
          Attachment 1888.v4.patch [ 12448506 ]
          Ravi Gummadi made changes -
          Status Reopened [ 4 ] Patch Available [ 10002 ]
          Hadoop Flags [Reviewed]
          Amareshwari Sriramadasu made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Amareshwari Sriramadasu made changes -
          Link This issue duplicates MAPREDUCE-1138 [ MAPREDUCE-1138 ]
          Konstantin Shvachko made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Ravi Gummadi
              Reporter:
              Amareshwari Sriramadasu
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development