Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10935

AM Total Queue Limit goes below per-user AM Limit if parent is full.

    XMLWordPrintableJSON

Details

    Description

      This happens when DRF is enabled and all of one resource is consumed but the second resources still has plenty available.

      This is reproduceable by setting up a parent queue where the capacity and max capacity are the same, with 2 or more sub-queues whose max capacity is 100%.

      In one of the sub-queues, start a long-running app that consumes all resources in the parent queue's hieararchy. This app will consume all of the memory but not vary many vcores (for example)

      In a second queue, submit an app. The Max Application Master Resources Per User limit is much more than the Max Application Master Resources limit.

      Attachments

        1. Screen Shot 2021-09-07 at 12.49.52 PM.png
          166 kB
          Eric Payne
        2. Screen Shot 2021-09-07 at 12.55.37 PM.png
          106 kB
          Eric Payne
        3. YARN-10935.001.patch
          2 kB
          Eric Payne
        4. YARN-10935.002.patch
          7 kB
          Eric Payne
        5. YARN-10935.003.patch
          7 kB
          Eric Payne
        6. YARN-10935.branch-2.10.003.patch
          8 kB
          Eric Payne
        7. YARN-10935.branch-3.2.003.patch
          7 kB
          Eric Payne

        Activity

          People

            epayne Eric Payne
            epayne Eric Payne
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: