Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1287

Avoid calling Partitioner with only 1 reducer

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 0.22.0
    • 0.21.0
    • None
    • None
    • Incompatible change, Reviewed
    • For jobs with only one reducer, the Partitioner will no longer be called. Applications depending on Partitioners modifying records for single reducer jobs will need to move this functionality elsewhere.

    Description

      Partitioners are currently called for each record, even though all are destined for the same reduce.

      Attachments

        1. M1287-4.patch
          2 kB
          Christopher Douglas
        2. M1287-5.patch
          2 kB
          Christopher Douglas
        3. M1287-6.patch
          7 kB
          Christopher Douglas
        4. MAPREDUCE-1287.2.patch
          1 kB
          Ed Mazur
        5. MAPREDUCE-1287.3.patch
          1 kB
          Ed Mazur
        6. MAPREDUCE-1287.patch
          0.7 kB
          Ed Mazur

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            cdouglas Christopher Douglas
            edmazur Ed Mazur
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment