Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10496 [Umbrella] Support Flexible Auto Queue Creation in Capacity Scheduler
  3. YARN-10620

fs2cs: parentQueue for certain placement rules are not set during conversion

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.4.0
    • 3.4.0
    • capacity scheduler
    • Reviewed

    Description

      There are some placement rules in FS which are currently not handled properly by fs2cs:

      <queuePlacementPolicy>
              <rule name="user" create="true"/>
              <rule name="primaryGroup" create="true"/>
          </queuePlacementPolicy>
      

      The first rule means that if the user queue doesn't exist, it should be created as root.<user>.
      The second means the same thing, except refers to the primary group instead of the submitting user: root.<primaryGroup>.

      The problem is that in order for the create="true" setting to take effect, we must set the parent queue in the generated JSON:

      Current:

      {
        "rules" : [ {
          "type" : "user",
          "matches" : "*",
          "policy" : "user",
          "fallbackResult" : "skip",
          "create" : true
        }, {
          "type" : "user",
          "matches" : "*",
          "policy" : "primaryGroup",
          "fallbackResult" : "skip",
          "create" : true
        } ]
      }
      

      Expected:

      {
        "rules" : [ {
          "type" : "user",
          "matches" : "*",
          "policy" : "user",
          "fallbackResult" : "skip",
          "parentQueue": "root",
          "create" : true
        }, {
          "type" : "user",
          "matches" : "*",
          "policy" : "primaryGroup",
          "fallbackResult" : "skip",
          "parentQueue": "root",
          "create" : true
        } ]
      

      This is missing right now and it need to be fixed.

      Attachments

        1. YARN-10620-001.patch
          4 kB
          Peter Bacsko
        2. YARN-10620-002.patch
          5 kB
          Peter Bacsko

        Activity

          People

            pbacsko Peter Bacsko
            pbacsko Peter Bacsko
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: