Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3310

Custom grouping comparator cannot be set for Combiners

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.20.1
    • 1.3.0, 2.3.0
    • client
    • None
    • All

    • Reviewed

    Description

      Combiners are often described as 'Reducers running on the Map side'.

      As Reducers, Combiners are fed <K,

      {V}>, where {V}

      is built by grouping values associated with the 'same' key.

      For Reducers, the comparator used for grouping values can be set independently of that used to sort the keys (using Job.setGroupingComparatorClass).

      Such a configuration is not possible for Combiners, meaning some things done in Reducers cannot be done in Combiners (such as secondary sort).

      It would be handy to have a Job.setCombinerGroupingComparatorClass method that would allow the setting of the grouping comparator used when applying a Combiner.

      Attachments

        1. MAPREDUCE-3310-trunk.patch
          29 kB
          Alejandro Abdelnur
        2. MAPREDUCE-3310-branch-1.patch
          21 kB
          Alejandro Abdelnur
        3. MAPREDUCE-3310-trunk.patch
          28 kB
          Alejandro Abdelnur
        4. MAPREDUCE-3310-branch-1.patch
          18 kB
          Alejandro Abdelnur
        5. MAPREDUCE-3310-trunk.patch
          28 kB
          Alejandro Abdelnur
        6. MAPREDUCE-3310-trunk.patch
          28 kB
          Alejandro Abdelnur

        Activity

          People

            tucu00 Alejandro Abdelnur
            herberts Mathias Herberts
            Votes:
            1 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: