Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5969

FairShareComparator: Cache value of getResourceUsage for better performance

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.7.1
    • 2.9.0, 3.0.0-alpha2
    • fairscheduler
    • None
    • Reviewed

    Description

      in FairShareComparator class, the performance of function getResourceUsage() is very poor. It will be executed above 100,000,000 times per second.
      In our scene, It takes 20 seconds per minute.
      A simple solution is to reduce call counts of the function.

      Attachments

        1. YARN-5969.patch
          3 kB
          zhangshilong
        2. pending_before.png
          123 kB
          zhangshilong
        3. pending_after.png
          130 kB
          zhangshilong
        4. containerAllocatedDelta_before.png
          184 kB
          zhangshilong
        5. containerAllocated_after.png
          212 kB
          zhangshilong
        6. apprunning_before.png
          164 kB
          zhangshilong
        7. apprunning_after.png
          184 kB
          zhangshilong
        8. 20161222.patch
          3 kB
          zhangshilong
        9. 20161206.patch
          2 kB
          zhangshilong

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            zsl2007 zhangshilong
            zsl2007 zhangshilong
            Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment