Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5221

Reduce side Combiner is not used when using the new API

    XMLWordPrintableJSON

Details

    • Bug
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • 2.0.4-alpha
    • None
    • None

    Description

      If a combiner is specified using o.a.h.mapreduce.Job.setCombinerClass - this will silently ignored on the reduce side since the reduce side usage is only aware of the old api combiner.
      This doesn't fail the job - since the new combiner key does not deprecate the old key.

      Attachments

        1. MAPREDUCE-5221.1.patch
          5 kB
          Tsuyoshi Ozawa
        2. MAPREDUCE-5221.10.patch
          21 kB
          Tsuyoshi Ozawa
        3. MAPREDUCE-5221.11.patch
          23 kB
          Tsuyoshi Ozawa
        4. MAPREDUCE-5221.2.patch
          5 kB
          Tsuyoshi Ozawa
        5. MAPREDUCE-5221.3.patch
          6 kB
          Tsuyoshi Ozawa
        6. MAPREDUCE-5221.4.patch
          4 kB
          Tsuyoshi Ozawa
        7. MAPREDUCE-5221.5.patch
          21 kB
          Tsuyoshi Ozawa
        8. MAPREDUCE-5221.6.patch
          22 kB
          Tsuyoshi Ozawa
        9. MAPREDUCE-5221.7.patch
          21 kB
          Tsuyoshi Ozawa
        10. MAPREDUCE-5221.7-2.patch
          21 kB
          Tsuyoshi Ozawa
        11. MAPREDUCE-5221.8.patch
          19 kB
          Tsuyoshi Ozawa
        12. MAPREDUCE-5221.9.patch
          21 kB
          Tsuyoshi Ozawa

        Issue Links

          is blocked by

          Sub-task - The sub-task of the issue MAPREDUCE-5294 Shuffle#MergeManager should support org.apache.hadoop.mapreduce.Reducer

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue MAPREDUCE-5295 ShuffleConsumerPlugin.Context should remove unused combiner-related methods and fields

          • Major - Major loss of function.
          • Resolved
          There are no Sub-Tasks for this issue.

          Activity

            People

              ozawa Tsuyoshi Ozawa
              sseth Siddharth Seth
              Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

                Created:
                Updated: