Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2632

Avoid calling the partitioner when the numReduceTasks is 1.

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.23.0
    • 3.0.0-alpha1
    • None
    • None
    • Incompatible change, Reviewed
    • A partitioner is now only created if there are multiple reducers.

    Description

      We can avoid the call to the partitioner when the number of reducers is 1.This will avoid the unnecessary computations by the partitioner.

      Attachments

        1. 0001-MAPREDUCE-2632.patch
          7 kB
          Sunil G
        2. MAPREDUCE-2632.patch
          3 kB
          Ravi Teja Ch N V
        3. MAPREDUCE-2632-1.patch
          9 kB
          Ravi Teja Ch N V
        4. mr-2632-2.patch
          6 kB
          Karthik Kambatla
        5. mr-2632-3.patch
          9 kB
          Karthik Kambatla
        6. mr-2632-4.patch
          11 kB
          Karthik Kambatla

        Issue Links

        incorporates

        Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-1287 Avoid calling Partitioner with only 1 reducer

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Closed

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            sunilg Sunil G
            raviteja Ravi Teja Ch N V
            Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment