Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.1
    • Component/s: mr-am, mrv2, performance
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Optimized Job's progress calculations in MR AM.

      Description

      Besides catering to client requests, Job progress is calculated in every heartbeat to the RM so as to print the MR AM's progress. Today the map and reduce progresses are calculated by looking up of each task in a big map while we can simply make do with a scan and aggregate. With large number of tasks, this can make a difference.

      1. MAPREDUCE-3568-20111215.1.txt
        11 kB
        Vinod Kumar Vavilapalli
      2. MAPREDUCE-3568-20111220.txt
        14 kB
        Vinod Kumar Vavilapalli
      3. MAPREDUCE-3568-20111222.txt
        32 kB
        Vinod Kumar Vavilapalli
      4. MAPREDUCE-3568-20111227.txt
        34 kB
        Vinod Kumar Vavilapalli

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Vinod Kumar Vavilapalli
            Reporter:
            Vinod Kumar Vavilapalli
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development