Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4606

CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps

    XMLWordPrintableJSON

Details

    Description

      Currently, if all applications belong to same user in LeafQueue are pending (caused by max-am-percent, etc.), ActiveUsersManager still considers the user is an active user. This could lead to starvation of active applications, for example:

      • App1(belongs to user1)/app2(belongs to user2) are active, app3(belongs to user3)/app4(belongs to user4) are pending
      • ActiveUsersManager returns #active-users=4
      • However, there're only two users (user1/user2) are able to allocate new resources. So computed user-limit-resource could be lower than expected.

      Attachments

        1. YARN-4606.007.patch
          11 kB
          Manikandan R
        2. YARN-4606.006.patch
          16 kB
          Manikandan R
        3. YARN-4606.005.patch
          16 kB
          Manikandan R
        4. YARN-4606.POC.3.patch
          3 kB
          Eric Payne
        5. YARN-4606.004.patch
          40 kB
          Manikandan R
        6. YARN-4606.003.patch
          32 kB
          Manikandan R
        7. YARN-4606.002.patch
          22 kB
          Manikandan R
        8. YARN-4606.001.patch
          23 kB
          Manikandan R
        9. YARN-4606.POC.2.patch
          19 kB
          Eric Payne
        10. YARN-4606.POC.patch
          14 kB
          Manikandan R
        11. YARN-4606.1.poc.patch
          14 kB
          Wangda Tan

        Issue Links

          Activity

            People

              manirajv06@gmail.com Manikandan R
              karams Karam Singh
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: