Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-291 [Umbrella] Dynamic resource configuration
  3. YARN-313

Add Admin API for supporting node resource configuration in command line

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: client, graceful
    • Labels:
      None
    • Target Version/s:
    • Release Note:
      Hide
      After this patch, the feature to support NM resource dynamically configuration is completed, so that user can configure NM with new resource without bring NM down or decommissioned.
      Two CLIs are provided to support update resources on individual node or a batch of nodes:
      1. Update resource on single node: yarn rmadmin -updateNodeResource [NodeID] [MemSize] [vCores]
      2. Update resource on a batch of nodes: yarn rmadmin -refreshNodesResources, that reflect nodes' resource configuration defined in dynamic-resources.xml which is loaded by RM dynamically (like capacity-scheduler.xml or fair-scheduler.xml).
      The first version of configuration format is:
      <configuration>
        <property>
          <name>yarn.resource.dynamic.nodes</name>
          <value>h1:1234</value>
        </property>
        <property>
          <name>yarn.resource.dynamic.h1:1234.vcores</name>
          <value>16</value>
        </property>
        <property>
          <name>yarn.resource.dynamic.h1:1234.memory</name>
          <value>1024</value>
        </property>
      </configuration>
      Show
      After this patch, the feature to support NM resource dynamically configuration is completed, so that user can configure NM with new resource without bring NM down or decommissioned. Two CLIs are provided to support update resources on individual node or a batch of nodes: 1. Update resource on single node: yarn rmadmin -updateNodeResource [NodeID] [MemSize] [vCores] 2. Update resource on a batch of nodes: yarn rmadmin -refreshNodesResources, that reflect nodes' resource configuration defined in dynamic-resources.xml which is loaded by RM dynamically (like capacity-scheduler.xml or fair-scheduler.xml). The first version of configuration format is: <configuration>   <property>     <name>yarn.resource.dynamic.nodes</name>     <value>h1:1234</value>   </property>   <property>     <name>yarn.resource.dynamic.h1:1234.vcores</name>     <value>16</value>   </property>   <property>     <name>yarn.resource.dynamic.h1:1234.memory</name>     <value>1024</value>   </property> </configuration>

      Description

      We should provide some admin interface, e.g. "yarn rmadmin -refreshResources" to support changes of node's resource specified in a config file.

        Attachments

        1. YARN-313-sample.patch
          40 kB
          Kenji Kikushima
        2. YARN-313-v1.patch
          50 kB
          Junping Du
        3. YARN-313-v2.patch
          52 kB
          Junping Du
        4. YARN-313-v3.patch
          53 kB
          Junping Du
        5. YARN-313-v4.patch
          53 kB
          Íñigo Goiri
        6. YARN-313-v5.patch
          54 kB
          Íñigo Goiri
        7. YARN-313-v6.patch
          54 kB
          Íñigo Goiri
        8. YARN-313-v7.patch
          53 kB
          Íñigo Goiri
        9. YARN-313-v8.patch
          53 kB
          Íñigo Goiri
        10. YARN-313-v9.patch
          53 kB
          Íñigo Goiri
        11. YARN-313-v10.patch
          57 kB
          Íñigo Goiri
        12. YARN-313-v11.patch
          57 kB
          Íñigo Goiri

          Issue Links

            Activity

              People

              • Assignee:
                elgoiri Íñigo Goiri
                Reporter:
                djp Junping Du
              • Votes:
                0 Vote for this issue
                Watchers:
                17 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: