Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1287

Avoid calling Partitioner with only 1 reducer

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      For jobs with only one reducer, the Partitioner will no longer be called. Applications depending on Partitioners modifying records for single reducer jobs will need to move this functionality elsewhere.

      Description

      Partitioners are currently called for each record, even though all are destined for the same reduce.

      1. MAPREDUCE-1287.patch
        0.7 kB
        Ed Mazur
      2. MAPREDUCE-1287.3.patch
        1 kB
        Ed Mazur
      3. MAPREDUCE-1287.2.patch
        1 kB
        Ed Mazur
      4. M1287-6.patch
        7 kB
        Chris Douglas
      5. M1287-5.patch
        2 kB
        Chris Douglas
      6. M1287-4.patch
        2 kB
        Chris Douglas

        Issue Links

          Activity

            People

            • Assignee:
              Chris Douglas
              Reporter:
              Ed Mazur
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development