Details

    • Sub-task
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 0.23.0
    • 0.23.1
    • mr-am, mrv2, performance
    • None
    • Reviewed
    • Optimized Job's progress calculations in MR AM.

    Description

      Besides catering to client requests, Job progress is calculated in every heartbeat to the RM so as to print the MR AM's progress. Today the map and reduce progresses are calculated by looking up of each task in a big map while we can simply make do with a scan and aggregate. With large number of tasks, this can make a difference.

      Attachments

        1. MAPREDUCE-3568-20111215.1.txt
          11 kB
          Vinod Kumar Vavilapalli
        2. MAPREDUCE-3568-20111220.txt
          14 kB
          Vinod Kumar Vavilapalli
        3. MAPREDUCE-3568-20111222.txt
          32 kB
          Vinod Kumar Vavilapalli
        4. MAPREDUCE-3568-20111227.txt
          34 kB
          Vinod Kumar Vavilapalli

        Activity

          People

            vinodkv Vinod Kumar Vavilapalli
            vinodkv Vinod Kumar Vavilapalli
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: