Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6242 [Umbrella] Miscellaneous Scheduler Performance Improvements
  3. YARN-4090

Make Collections.sort() more efficient by caching resource usage

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.8.1, 3.0.0-alpha3
    • 2.9.0, 3.0.0, 3.1.0
    • fairscheduler
    • None
    • Reviewed

    Description

      Collections.sort() consumes too much time in a scheduling round.

      Attachments

        1. sampling1.jpg
          320 kB
          Xianyin Xin
        2. sampling2.jpg
          285 kB
          Xianyin Xin
        3. YARN-4090.001.patch
          10 kB
          Xianyin Xin
        4. YARN-4090.002.patch
          10 kB
          Xianyin Xin
        5. YARN-4090.003.patch
          10 kB
          Xianyin Xin
        6. YARN-4090.004.patch
          6 kB
          zhangshilong
        7. YARN-4090.005.patch
          7 kB
          zhangshilong
        8. YARN-4090.006.patch
          7 kB
          zhangshilong
        9. YARN-4090.007.patch
          13 kB
          Yufei Gu
        10. YARN-4090.008.patch
          11 kB
          Yufei Gu
        11. YARN-4090.009.patch
          11 kB
          Yufei Gu
        12. YARN-4090-preview.patch
          6 kB
          Xianyin Xin
        13. YARN-4090-TestResult.pdf
          783 kB
          Xianyin Xin

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. YARN-7385 TestFairScheduler#testUpdateDemand and TestFSLeafQueue#testUpdateDemand are failing with NPE

          • Major - Major loss of function.
          • Resolved
          duplicates

          Improvement - An improvement or enhancement to an existing feature or task. YARN-1297 FairScheduler: Move some logs to debug and check if debug logging is enabled

          • Major - Major loss of function.
          • Resolved
          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. YARN-6947 The implementation of Schedulable#getResourceUsage so inefficiency that can reduce the performance of scheduling

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. YARN-4691 Cache resource usage at FSLeafQueue level

          • Major - Major loss of function.
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. YARN-5479 FairScheduler: Scheduling performance improvement

          • Major - Major loss of function.
          • Open
          relates to

          Bug - A problem which impairs or prevents the functions of the product. YARN-4120 FSAppAttempt.getResourceUsage() should not take preemptedResource into account

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              yufeigu Yufei Gu
              xinxianyin Xianyin Xin
              Votes:
              2 Vote for this issue
              Watchers:
              29 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: