Hadoop YARN
  1. Hadoop YARN
  2. YARN-1160

allow admins to force app deployment on a specific host

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 3.0.0
    • Fix Version/s: None
    • Component/s: resourcemanager
    • Labels:
      None

      Description

      Currently you ask YARN to get slots on a host and it finds a slot on that machine -or, if unavailable or there is no room, on a host nearby as far as the topology is concerned.

      People with admin rights should have the option to deploy a process on a specific host and have it run there even if there are no free slots -and to fail if the machine is not available. This would let you deploy admin-specific process across a cluster.

        Issue Links

          Activity

          Hide
          Alejandro Abdelnur added a comment -

          You can already ask for container exactly on a specific node setting relaxLocality to FALSE in the ResourceRequest. Though, this does not allow you to get a container if there is no capacity in the node.

          Show
          Alejandro Abdelnur added a comment - You can already ask for container exactly on a specific node setting relaxLocality to FALSE in the ResourceRequest. Though, this does not allow you to get a container if there is no capacity in the node.
          Hide
          Steve Loughran added a comment -

          -Yes, and if you don't get that container it just stays in the queue -no notification to the AM. This is about being able to force things in without that wait and irrespective of space

          Show
          Steve Loughran added a comment - -Yes, and if you don't get that container it just stays in the queue -no notification to the AM. This is about being able to force things in without that wait and irrespective of space
          Hide
          Alejandro Abdelnur added a comment -

          If I get it correctly, what you want is use Yarn to run a process in a particular host completely outside of the resource pool.

          Show
          Alejandro Abdelnur added a comment - If I get it correctly, what you want is use Yarn to run a process in a particular host completely outside of the resource pool.
          Hide
          Steve Loughran added a comment -

          yes. use case is deploying services cluster wide that use HDFS for provisioning and an AM to choreograph rollout, react to failures & addition of new nodes in the cluster

          Show
          Steve Loughran added a comment - yes. use case is deploying services cluster wide that use HDFS for provisioning and an AM to choreograph rollout, react to failures & addition of new nodes in the cluster
          Hide
          Alejandro Abdelnur added a comment -

          wouldn't make sense, instead having to deal with admin rights, to use a special queue for this and anybody in the queue ACL could manage such services?

          Show
          Alejandro Abdelnur added a comment - wouldn't make sense, instead having to deal with admin rights, to use a special queue for this and anybody in the queue ACL could manage such services?
          Hide
          Steve Loughran added a comment -

          -seems reasonable. The key thing is to ignore resource allocations, both in the demand of the app and the containers deployed -which lets people deploy static applications across the cluster, using YARN to push out the binaries & report failures to an AM

          Show
          Steve Loughran added a comment - -seems reasonable. The key thing is to ignore resource allocations, both in the demand of the app and the containers deployed -which lets people deploy static applications across the cluster, using YARN to push out the binaries & report failures to an AM

            People

            • Assignee:
              Unassigned
              Reporter:
              Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:

                Development