Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-291 [Umbrella] Dynamic resource configuration
  3. YARN-313

Add Admin API for supporting node resource configuration in command line

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • 2.8.0, 3.0.0-alpha1
    • client, graceful
    • None
    • Hide
      After this patch, the feature to support NM resource dynamically configuration is completed, so that user can configure NM with new resource without bring NM down or decommissioned.
      Two CLIs are provided to support update resources on individual node or a batch of nodes:
      1. Update resource on single node: yarn rmadmin -updateNodeResource [NodeID] [MemSize] [vCores]
      2. Update resource on a batch of nodes: yarn rmadmin -refreshNodesResources, that reflect nodes' resource configuration defined in dynamic-resources.xml which is loaded by RM dynamically (like capacity-scheduler.xml or fair-scheduler.xml).
      The first version of configuration format is:
      <configuration>
        <property>
          <name>yarn.resource.dynamic.nodes</name>
          <value>h1:1234</value>
        </property>
        <property>
          <name>yarn.resource.dynamic.h1:1234.vcores</name>
          <value>16</value>
        </property>
        <property>
          <name>yarn.resource.dynamic.h1:1234.memory</name>
          <value>1024</value>
        </property>
      </configuration>
      Show
      After this patch, the feature to support NM resource dynamically configuration is completed, so that user can configure NM with new resource without bring NM down or decommissioned. Two CLIs are provided to support update resources on individual node or a batch of nodes: 1. Update resource on single node: yarn rmadmin -updateNodeResource [NodeID] [MemSize] [vCores] 2. Update resource on a batch of nodes: yarn rmadmin -refreshNodesResources, that reflect nodes' resource configuration defined in dynamic-resources.xml which is loaded by RM dynamically (like capacity-scheduler.xml or fair-scheduler.xml). The first version of configuration format is: <configuration>   <property>     <name>yarn.resource.dynamic.nodes</name>     <value>h1:1234</value>   </property>   <property>     <name>yarn.resource.dynamic.h1:1234.vcores</name>     <value>16</value>   </property>   <property>     <name>yarn.resource.dynamic.h1:1234.memory</name>     <value>1024</value>   </property> </configuration>

    Description

      We should provide some admin interface, e.g. "yarn rmadmin -refreshResources" to support changes of node's resource specified in a config file.

      Attachments

        1. YARN-313-sample.patch
          40 kB
          Kenji Kikushima
        2. YARN-313-v1.patch
          50 kB
          Junping Du
        3. YARN-313-v2.patch
          52 kB
          Junping Du
        4. YARN-313-v3.patch
          53 kB
          Junping Du
        5. YARN-313-v4.patch
          53 kB
          Íñigo Goiri
        6. YARN-313-v5.patch
          54 kB
          Íñigo Goiri
        7. YARN-313-v6.patch
          54 kB
          Íñigo Goiri
        8. YARN-313-v7.patch
          53 kB
          Íñigo Goiri
        9. YARN-313-v8.patch
          53 kB
          Íñigo Goiri
        10. YARN-313-v9.patch
          53 kB
          Íñigo Goiri
        11. YARN-313-v10.patch
          57 kB
          Íñigo Goiri
        12. YARN-313-v11.patch
          57 kB
          Íñigo Goiri

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            elgoiri Íñigo Goiri
            junping_du Junping Du
            Votes:
            0 Vote for this issue
            Watchers:
            18 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment