Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3306

[Umbrella] Proposing per-queue Policy driven scheduling in YARN

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • scheduler
    • None

    Description

      Scheduling layout in Apache Hadoop YARN today is very coarse grained. This proposal aims at converting today’s rigid scheduling in YARN to a per­-queue policy driven architecture.

      We propose the creation of a c​ommon policy framework​ and implement a​common set of policies​ that administrators can pick and chose per queue

      • Make scheduling policies configurable per queue
      • Initially, we limit ourselves to a new type of scheduling policy that determines the ordering of applications within the l​eaf ­queue
      • In the near future, we will also pursue parent­ queue level policies and potential algorithm reuse through a separate type of policies that control resource limits per queue, user, application etc.

      Attachments

        1. PerQueuePolicydrivenschedulinginYARN.pdf
          247 kB
          Vinod Kumar Vavilapalli

        Activity

          People

            vinodkv Vinod Kumar Vavilapalli
            vinodkv Vinod Kumar Vavilapalli
            Votes:
            1 Vote for this issue
            Watchers:
            30 Start watching this issue

            Dates

              Created:
              Updated: