Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5969

FairShareComparator: Cache value of getResourceUsage for better performance

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: fairscheduler
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      in FairShareComparator class, the performance of function getResourceUsage() is very poor. It will be executed above 100,000,000 times per second.
      In our scene, It takes 20 seconds per minute.
      A simple solution is to reduce call counts of the function.

        Attachments

        1. 20161206.patch
          2 kB
          zhangshilong
        2. 20161222.patch
          3 kB
          zhangshilong
        3. apprunning_after.png
          184 kB
          zhangshilong
        4. apprunning_before.png
          164 kB
          zhangshilong
        5. containerAllocated_after.png
          212 kB
          zhangshilong
        6. containerAllocatedDelta_before.png
          184 kB
          zhangshilong
        7. pending_after.png
          130 kB
          zhangshilong
        8. pending_before.png
          123 kB
          zhangshilong
        9. YARN-5969.patch
          3 kB
          zhangshilong

          Activity

            People

            • Assignee:
              zsl2007 zhangshilong
              Reporter:
              zsl2007 zhangshilong

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment