Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10496 [Umbrella] Support Flexible Auto Queue Creation in Capacity Scheduler
  3. YARN-10564

Support Auto Queue Creation template configurations

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.4.0
    • 3.4.0
    • capacity scheduler
    • None
    • Reviewed

    Description

      Similar to how the template configuration works for ManagedParents, we need to support templates for the new auto queue creation logic. Proposition is to allow wildcards in template configs such as:

      yarn.scheduler.capacity.root.*.*.weight 10

      which would mean, that set weight to 10 of every leaf of every parent under root.
      We should possibly take an approach, that could support arbitrary depth of template configuration, because we might need to lift the limitation of auto queue nesting.

      Attachments

        1. YARN-10564.001.patch
          13 kB
          Andras Gyori
        2. YARN-10564.002.patch
          15 kB
          Andras Gyori
        3. YARN-10564.003.patch
          22 kB
          Andras Gyori
        4. YARN-10564.004.patch
          23 kB
          Qi Zhu
        5. YARN-10564.005.patch
          21 kB
          Andras Gyori
        6. YARN-10564.006.patch
          25 kB
          Andras Gyori
        7. YARN-10564.poc.001.patch
          70 kB
          Qi Zhu

        Issue Links

          is duplicated by

          Sub-task - The sub-task of the issue YARN-10645 Fix queue state related update for auto created queue.

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            People

              gandras Andras Gyori
              gandras Andras Gyori
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: