Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3310

Custom grouping comparator cannot be set for Combiners

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.1
    • Fix Version/s: 1.3.0, 2.3.0
    • Component/s: client
    • Labels:
      None
    • Environment:

      All

    • Hadoop Flags:
      Reviewed

      Description

      Combiners are often described as 'Reducers running on the Map side'.

      As Reducers, Combiners are fed <K,

      {V}>, where {V}

      is built by grouping values associated with the 'same' key.

      For Reducers, the comparator used for grouping values can be set independently of that used to sort the keys (using Job.setGroupingComparatorClass).

      Such a configuration is not possible for Combiners, meaning some things done in Reducers cannot be done in Combiners (such as secondary sort).

      It would be handy to have a Job.setCombinerGroupingComparatorClass method that would allow the setting of the grouping comparator used when applying a Combiner.

        Attachments

        1. MAPREDUCE-3310-branch-1.patch
          21 kB
          Alejandro Abdelnur
        2. MAPREDUCE-3310-branch-1.patch
          18 kB
          Alejandro Abdelnur
        3. MAPREDUCE-3310-trunk.patch
          29 kB
          Alejandro Abdelnur
        4. MAPREDUCE-3310-trunk.patch
          28 kB
          Alejandro Abdelnur
        5. MAPREDUCE-3310-trunk.patch
          28 kB
          Alejandro Abdelnur
        6. MAPREDUCE-3310-trunk.patch
          28 kB
          Alejandro Abdelnur

          Activity

            People

            • Assignee:
              tucu00 Alejandro Abdelnur
              Reporter:
              herberts Mathias Herberts
            • Votes:
              1 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: