[YARN-11514] Extend SchedulerResponse with capacityVector

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Labels: pull-request-available
• Hadoop Flags: Reviewed
• Assignee: Benjamin Teke
• Reporter: Tamas Domok
• Created: 15/Jun/23 07:31
• Updated: 25/Sep/23 14:24
• Resolved: 25/Sep/23 14:24

Description

The goal is to add the capacityVector to the Scheduler response (XML/JSON).

• CapacitySchedulerQueueInfo.java
• PartitionQueueCapacitiesInfo.java

The proposed format in the design doc (YARN-10888):

{
   "capacityVector": {
      "memory-mb": "30%",
      "vcores": "16"
   }
}

<?xml version="1.0" encoding="UTF-8" ?>
<capacityVector>
  <memory-mb>30%</memory-mb>
  <vcores>16</vcores>
</capacityVector>

Unfortunately the current jsonProvider (MoxyJsonFeature or JettisonFeature not sure) serialise map structures in the following way:

{
  "capacityVector":{
    "entry":[
      {
        "key":"memory-mb",
        "value":"12288"
      },
      {
        "key":"vcores",
        "value":"86%"
      }
    ]
  }
}

<?xml version="1.0" encoding="UTF-8"?>
<capacityVector>
   <entry>
      <key>memory-mb</key>
      <value>1288</value>
   </entry>
   <entry>
      <key>vcores</key>
      <value>12</value>
   </entry>
</capacityVector>

Based on some research with the following two dependencies we could achieve the proposed format:

• jersey-media-json-jackson (this one is used in the apps catalog already)
• jackson-dataformat-xml

Some concerns:

• 2 more dependencies
• for the XML when the content depends on the runtime content of the map is not XSD friendly
• name is capacityVector but it's represented in a map

An alternative could be to just store the capacityVector as a string, but then clients needs to parse it, and it's not particularly nice either:

{
   "capacityVector": "[\"memory-mb\":  12288, \"vcores\":  86%]"
}

<?xml version="1.0" encoding="UTF-8" ?>
<capacityVector>[&quot;memory-mb&quot;:  12288, &quot;vcores&quot;:  86%]</capacityVector>

Issue Links

• links to: GitHub Pull Request #5989