Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.3.0
    • Fix Version/s: 3.3.0, 3.2.2, 3.1.4
    • Component/s: capacity scheduler
    • Labels:
      None

      Description

      Queue Mutation API does not allow to remove a config. When removing a node label from a queue and it's capacity config

      
       <sched-conf>
          <update-queue>
            <queue-name>root.batch</queue-name>
            <params>
              <entry>
                <key>accessible-node-labels</key>
                <value></value>
              </entry>
              <entry>
                <key>accessible-node-labels.x.capacity</key>
                <value></value>
              </entry>
            </params>
          </update-queue>
        </sched-conf>
      

      It fails with below.

      Caused by: java.lang.NumberFormatException: empty String
      	at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1842)
      	at sun.misc.FloatingDecimal.parseFloat(FloatingDecimal.java:122)
      	at java.lang.Float.parseFloat(Float.java:451)
      	at org.apache.hadoop.conf.Configuration.getFloat(Configuration.java:1632)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.internalGetLabeledQueueCapacity(CapacitySchedulerConfiguration.java:682)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getLabeledQueueCapacity(CapacitySchedulerConfiguration.java:697)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueUtils.loadCapacitiesByLabelsFromConf(CSQueueUtils.java:136)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueUtils.loadUpdateAndCheckCapacities(CSQueueUtils.java:111)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.AbstractCSQueue.setupConfigurableCapacities(AbstractCSQueue.java:185)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.AbstractCSQueue.setupQueueConfigs(AbstractCSQueue.java:362)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue.setupQueueConfigs(LeafQueue.java:172)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue.<init>(LeafQueue.java:157)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue.<init>(LeafQueue.java:139)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.parseQueue(CapacitySchedulerQueueManager.java:259)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.parseQueue(CapacitySchedulerQueueManager.java:283)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.reinitializeQueues(CapacitySchedulerQueueManager.java:171)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:785)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:497)
      	... 72 more
      

      FIX:

      Specifying below in update-queue will remove the capacity config.

           <entry>
                <key>accessible-node-labels.x.capacity</key>
                <value></value>
              </entry>
      

        Attachments

        1. YARN-9791-002.patch
          5 kB
          Prabhu Joseph
        2. YARN-9791-001.patch
          4 kB
          Prabhu Joseph

          Activity

            People

            • Assignee:
              prabhujoseph Prabhu Joseph
              Reporter:
              prabhujoseph Prabhu Joseph
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: