Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3306

[Umbrella] Proposing per-queue Policy driven scheduling in YARN

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: scheduler
    • Labels:
      None

      Description

      Scheduling layout in Apache Hadoop YARN today is very coarse grained. This proposal aims at converting today’s rigid scheduling in YARN to a per­-queue policy driven architecture.

      We propose the creation of a c​ommon policy framework​ and implement a​common set of policies​ that administrators can pick and chose per queue

      • Make scheduling policies configurable per queue
      • Initially, we limit ourselves to a new type of scheduling policy that determines the ordering of applications within the l​eaf ­queue
      • In the near future, we will also pursue parent­ queue level policies and potential algorithm reuse through a separate type of policies that control resource limits per queue, user, application etc.

        Attachments

        1. PerQueuePolicydrivenschedulinginYARN.pdf
          247 kB
          Vinod Kumar Vavilapalli

          Activity

            People

            • Assignee:
              vinodkv Vinod Kumar Vavilapalli
              Reporter:
              vinodkv Vinod Kumar Vavilapalli
            • Votes:
              1 Vote for this issue
              Watchers:
              30 Start watching this issue

              Dates

              • Created:
                Updated: