Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.1
    • Component/s: mr-am, mrv2, performance
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Optimized Job's progress calculations in MR AM.

      Description

      Besides catering to client requests, Job progress is calculated in every heartbeat to the RM so as to print the MR AM's progress. Today the map and reduce progresses are calculated by looking up of each task in a big map while we can simply make do with a scan and aggregate. With large number of tasks, this can make a difference.

        Attachments

        1. MAPREDUCE-3568-20111227.txt
          34 kB
          Vinod Kumar Vavilapalli
        2. MAPREDUCE-3568-20111222.txt
          32 kB
          Vinod Kumar Vavilapalli
        3. MAPREDUCE-3568-20111220.txt
          14 kB
          Vinod Kumar Vavilapalli
        4. MAPREDUCE-3568-20111215.1.txt
          11 kB
          Vinod Kumar Vavilapalli

          Activity

            People

            • Assignee:
              vinodkv Vinod Kumar Vavilapalli
              Reporter:
              vinodkv Vinod Kumar Vavilapalli
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: