Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6151

FS preemption does not consider child queues over fairshare if the parent is under

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.8.0
    • 2.8.0
    • fairscheduler
    • None
    • Reviewed

    Description

      This is preemption bug happens before 2.8.0, which also described in YARN-3405.

      Queue hierarchy described as below:

                        root
                     /         \
             queue-1          queue-2       
                /      \
      queue-1-1     queue-1-2
      

      Assume cluster resource is 100 and all queues have same weights.

      1. queue-1-1 and queue-2 has apps. Each get 50 usage and 50 fairshare.
      2. When queue-1-2 is active, supposedly it will preempt 25 from queue-1-1, but this doesn't happen because preemption happens top-down, queue-2 could be the preemption candidate as long as queue-2 is less needy than queue-1, and queue-2 doesn't exceed the fair share which means preemption won't happen.

      We need to filter out queue-2 since it isn't a valid candidate.

      Attachments

        1. YARN-6151.branch-2.8.001.patch
          6 kB
          Yufei Gu
        2. YARN-6151.branch-2.8.002.patch
          9 kB
          Yufei Gu
        3. YARN-6151.branch-2.8.003.patch
          11 kB
          Yufei Gu

        Issue Links

          Activity

            People

              yufeigu Yufei Gu
              yufeigu Yufei Gu
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: