Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1287

Avoid calling Partitioner with only 1 reducer

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      For jobs with only one reducer, the Partitioner will no longer be called. Applications depending on Partitioners modifying records for single reducer jobs will need to move this functionality elsewhere.

      Description

      Partitioners are currently called for each record, even though all are destined for the same reduce.

        Attachments

        1. MAPREDUCE-1287.patch
          0.7 kB
          Ed Mazur
        2. MAPREDUCE-1287.3.patch
          1 kB
          Ed Mazur
        3. MAPREDUCE-1287.2.patch
          1 kB
          Ed Mazur
        4. M1287-6.patch
          7 kB
          Chris Douglas
        5. M1287-5.patch
          2 kB
          Chris Douglas
        6. M1287-4.patch
          2 kB
          Chris Douglas

          Issue Links

            Activity

              People

              • Assignee:
                chris.douglas Chris Douglas
                Reporter:
                edmazur Ed Mazur
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: