Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3561 [Umbrella ticket] Performance issues in YARN+MR
  3. MAPREDUCE-3566

MR AM slows down due to repeatedly constructing ContainerLaunchContext

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 0.23.0
    • 0.23.1
    • mr-am, mrv2
    • None

    Description

      The construction of the context is expensive, includes per-task trips to NameNode for obtaining the information about job.jar, job splits etc which is redundant across all tasks.

      We should have a common job-level context and a task-specific context inheriting from the common job-level context.

      Attachments

        1. MAPREDUCE-3566.patch
          27 kB
          Arun Murthy
        2. MAPREDUCE-3566-20111215.txt
          12 kB
          Vinod Kumar Vavilapalli
        3. MAPREDUCE-3566-20111220.txt
          15 kB
          Vinod Kumar Vavilapalli
        4. MAPREDUCE-3566-20120103.1.txt
          25 kB
          Vinod Kumar Vavilapalli
        5. MAPREDUCE-3566-20120103.txt
          25 kB
          Vinod Kumar Vavilapalli

        Activity

          People

            vinodkv Vinod Kumar Vavilapalli
            vinodkv Vinod Kumar Vavilapalli
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: