Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7070

Batch calls to RackResolver from AM

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Patch Available
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None
    • Target Version/s:
    • Flags:
      Patch

      Description

      TaskAttemptImpl calls RackResolver once for every split location. This results in many shells being spawned by the Application Master when the topology script is enabled. The worst case scenario would entail one call to the topology script for each Datanode. The NameNode is also topology aware, and uses batch calls to the topology script to improve performance.

        Attachments

        1. MAPREDUCE-7070.patch
          5 kB
          Greg Phillips

        Issue Links

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              gphillips Greg Phillips

              Dates

              • Created:
                Updated:

                Issue deployment