Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4606

CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps

    Details

      Description

      Currently, if all applications belong to same user in LeafQueue are pending (caused by max-am-percent, etc.), ActiveUsersManager still considers the user is an active user. This could lead to starvation of active applications, for example:

      • App1(belongs to user1)/app2(belongs to user2) are active, app3(belongs to user3)/app4(belongs to user4) are pending
      • ActiveUsersManager returns #active-users=4
      • However, there're only two users (user1/user2) are able to allocate new resources. So computed user-limit-resource could be lower than expected.

        Attachments

        1. YARN-4606.001.patch
          23 kB
          Manikandan R
        2. YARN-4606.002.patch
          22 kB
          Manikandan R
        3. YARN-4606.003.patch
          32 kB
          Manikandan R
        4. YARN-4606.004.patch
          40 kB
          Manikandan R
        5. YARN-4606.005.patch
          16 kB
          Manikandan R
        6. YARN-4606.006.patch
          16 kB
          Manikandan R
        7. YARN-4606.007.patch
          11 kB
          Manikandan R
        8. YARN-4606.1.poc.patch
          14 kB
          Wangda Tan
        9. YARN-4606.POC.2.patch
          19 kB
          Eric Payne
        10. YARN-4606.POC.3.patch
          3 kB
          Eric Payne
        11. YARN-4606.POC.patch
          14 kB
          Manikandan R

          Issue Links

            Activity

              People

              • Assignee:
                manirajv06@gmail.com Manikandan R
                Reporter:
                karams Karam Singh
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: