Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5969

FairShareComparator: Cache value of getResourceUsage for better performance

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: fairscheduler
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      in FairShareComparator class, the performance of function getResourceUsage() is very poor. It will be executed above 100,000,000 times per second.
      In our scene, It takes 20 seconds per minute.
      A simple solution is to reduce call counts of the function.

        Attachments

        1. 20161206.patch
          2 kB
          zhangshilong
        2. 20161222.patch
          3 kB
          zhangshilong
        3. apprunning_after.png
          184 kB
          zhangshilong
        4. apprunning_before.png
          164 kB
          zhangshilong
        5. containerAllocated_after.png
          212 kB
          zhangshilong
        6. containerAllocatedDelta_before.png
          184 kB
          zhangshilong
        7. pending_after.png
          130 kB
          zhangshilong
        8. pending_before.png
          123 kB
          zhangshilong
        9. YARN-5969.patch
          3 kB
          zhangshilong

          Activity

            People

            • Assignee:
              zsl2007 zhangshilong
              Reporter:
              zsl2007 zhangshilong
            • Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: