Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4963

capacity scheduler: Make number of OFF_SWITCH assignments per heartbeat configurable

    Details

    • Hadoop Flags:
      Reviewed

      Description

      Currently the capacity scheduler will allow exactly 1 OFF_SWITCH assignment per heartbeat. With more and more non MapReduce workloads coming along, the degree of locality is declining, causing scheduling to be significantly slower. It's still important to limit the number of OFF_SWITCH assignments to avoid densely packing OFF_SWITCH containers onto nodes.

      Proposal is to add a simple config that makes the number of OFF_SWITCH assignments configurable.

      Will upload candidate patch shortly.

      1. YARN-4963.001.patch
        12 kB
        Nathan Roberts
      2. YARN-4963.002.patch
        12 kB
        Nathan Roberts
      3. YARN-4963.003.patch
        12 kB
        Nathan Roberts
      4. YARN-4963.004.patch
        12 kB
        Nathan Roberts

        Issue Links

          Activity

          Hide
          leftnoteasy Wangda Tan added a comment -

          Nathan Roberts, +1 to this feature, this gonna be very useful.

          Instead of only limit #off-switch containers allocate per heartbeat, can we limit #containers allocation regardless of locality? I can see some values to limit #containers for rack/node locality as well. For example, if user wants their containers allocated spread across all the cluster.

          Show
          leftnoteasy Wangda Tan added a comment - Nathan Roberts , +1 to this feature, this gonna be very useful. Instead of only limit #off-switch containers allocate per heartbeat, can we limit #containers allocation regardless of locality? I can see some values to limit #containers for rack/node locality as well. For example, if user wants their containers allocated spread across all the cluster.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 1s trunk passed
          +1 compile 0m 29s trunk passed with JDK v1.8.0_77
          +1 compile 0m 29s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 19s trunk passed
          +1 mvnsite 0m 35s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 findbugs 1m 3s trunk passed
          +1 javadoc 0m 21s trunk passed with JDK v1.8.0_77
          +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 30s the patch passed
          +1 compile 0m 24s the patch passed with JDK v1.8.0_77
          +1 javac 0m 24s the patch passed
          +1 compile 0m 27s the patch passed with JDK v1.7.0_95
          +1 javac 0m 27s the patch passed
          -1 checkstyle 0m 17s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 1 new + 111 unchanged - 0 fixed = 112 total (was 111)
          +1 mvnsite 0m 32s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 findbugs 1m 13s the patch passed
          +1 javadoc 0m 19s the patch passed with JDK v1.8.0_77
          +1 javadoc 0m 24s the patch passed with JDK v1.7.0_95
          -1 unit 76m 39s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77.
          -1 unit 55m 24s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 22s Patch does not generate ASF License warnings.
          149m 4s



          Reason Tests
          JDK v1.8.0_77 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.webapp.TestRMWithCSRFFilter
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
          JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.webapp.TestRMWithCSRFFilter
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798973/YARN-4963.001.patch
          JIRA Issue YARN-4963
          Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 3e6608b4e1c2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6e6b6dd
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11097/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11097/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11097/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11097/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11097/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11097/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11097/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 1s trunk passed +1 compile 0m 29s trunk passed with JDK v1.8.0_77 +1 compile 0m 29s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 19s trunk passed +1 mvnsite 0m 35s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 3s trunk passed +1 javadoc 0m 21s trunk passed with JDK v1.8.0_77 +1 javadoc 0m 26s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 30s the patch passed +1 compile 0m 24s the patch passed with JDK v1.8.0_77 +1 javac 0m 24s the patch passed +1 compile 0m 27s the patch passed with JDK v1.7.0_95 +1 javac 0m 27s the patch passed -1 checkstyle 0m 17s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: patch generated 1 new + 111 unchanged - 0 fixed = 112 total (was 111) +1 mvnsite 0m 32s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 1m 13s the patch passed +1 javadoc 0m 19s the patch passed with JDK v1.8.0_77 +1 javadoc 0m 24s the patch passed with JDK v1.7.0_95 -1 unit 76m 39s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77. -1 unit 55m 24s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 149m 4s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.webapp.TestRMWithCSRFFilter   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.webapp.TestRMWithCSRFFilter   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798973/YARN-4963.001.patch JIRA Issue YARN-4963 Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 3e6608b4e1c2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6e6b6dd Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11097/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11097/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11097/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11097/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11097/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11097/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/11097/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          nroberts Nathan Roberts added a comment -

          Thanks Wangda Tan for the feedback. I agree that it would be a useful feature to be able to give some applications better spread, regardless of allocation type. Now we just have to figure out how to get there.

          My concern is that I don't think we'd want to implement it using the same simple approach if it's going to apply to all container types. For example, in our case we almost always want NODE_LOCAL and RACK_LOCAL to get scheduled as quickly as possible so I'd want the limit to be high, as opposed to OFF_SWITCH where I want the limit to be 3-5 to keep a nice balance between scheduling performance and clustering.

          The reason this check was introduced in the first place (iirc) was to prevent network-heavy applications from loading up on specific nodes. The OFF_SWITCH check was a simple way of achieving this at a global level. The feature I think you're asking for (please correct me if I misunderstood) is that applications should be able to request that container spread be prioritized over timely scheduling (kind of like locality delay does today). I completely agree this would be a useful knob for applications to have. It is a trade-off though. An application that wants really good spread would be sacrificing scheduling opportunities that would probably be given to applications behind them in the queue (like locality delay).

          So maybe there are two things to do:
          1) Have the global OFF_SWITCH check to handle the simple case of avoiding too many network-heavy applications on a node.
          2) A feature where applications can specify a max_containers_assigned_per_node_per_heartbeat. I think this would be checked down in LeafQueue.assignContainers().

          Even with #2 in place, I don't think #1 could immediately go away because the network-heavy applications would need to start properly specifying this limit.

          The other approach to get rid of #1 would be when network is a resource. Such applications could then request lots of network resource, which should prevent clustering.

          Does that make any sort of sense?

          Show
          nroberts Nathan Roberts added a comment - Thanks Wangda Tan for the feedback. I agree that it would be a useful feature to be able to give some applications better spread, regardless of allocation type. Now we just have to figure out how to get there. My concern is that I don't think we'd want to implement it using the same simple approach if it's going to apply to all container types. For example, in our case we almost always want NODE_LOCAL and RACK_LOCAL to get scheduled as quickly as possible so I'd want the limit to be high, as opposed to OFF_SWITCH where I want the limit to be 3-5 to keep a nice balance between scheduling performance and clustering. The reason this check was introduced in the first place (iirc) was to prevent network-heavy applications from loading up on specific nodes. The OFF_SWITCH check was a simple way of achieving this at a global level. The feature I think you're asking for (please correct me if I misunderstood) is that applications should be able to request that container spread be prioritized over timely scheduling (kind of like locality delay does today). I completely agree this would be a useful knob for applications to have. It is a trade-off though. An application that wants really good spread would be sacrificing scheduling opportunities that would probably be given to applications behind them in the queue (like locality delay). So maybe there are two things to do: 1) Have the global OFF_SWITCH check to handle the simple case of avoiding too many network-heavy applications on a node. 2) A feature where applications can specify a max_containers_assigned_per_node_per_heartbeat. I think this would be checked down in LeafQueue.assignContainers(). Even with #2 in place, I don't think #1 could immediately go away because the network-heavy applications would need to start properly specifying this limit. The other approach to get rid of #1 would be when network is a resource. Such applications could then request lots of network resource, which should prevent clustering. Does that make any sort of sense?
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Thanks Nathan Roberts for initiating discussion on this. We have seen off_switch assignment issue in large cluster as you described. Especially when cluster is running fully occupied resource, and container release happens all together from one node, only 1 container is assigned this node. This makes user thinks that why assignment is not happening even though resource is free.
          IMO, I think application specific configurations should be there rather at scheduler level. Some applications are fine with assigning containers in off_switch they can specify number of containers to be assigned. But few applications are very strict to node locality, they can configure 1 in off_switch.
          Thoughts?

          Show
          rohithsharma Rohith Sharma K S added a comment - Thanks Nathan Roberts for initiating discussion on this. We have seen off_switch assignment issue in large cluster as you described. Especially when cluster is running fully occupied resource, and container release happens all together from one node, only 1 container is assigned this node. This makes user thinks that why assignment is not happening even though resource is free. IMO, I think application specific configurations should be there rather at scheduler level. Some applications are fine with assigning containers in off_switch they can specify number of containers to be assigned. But few applications are very strict to node locality, they can configure 1 in off_switch. Thoughts?
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          I think application specific configurations should be there rather at scheduler level.

          Even i feel the same, any specfic reason it has been set only at the scheduler level other than the AMRM interface change ? We can keep the default value as 1 so that its still compatible. Also anyway allocation happens within app's & queue's capacity limits so i feel it would be ideal for app to decide how many allocations in off_switch node. thoughts ?

          Show
          Naganarasimha Naganarasimha G R added a comment - I think application specific configurations should be there rather at scheduler level. Even i feel the same, any specfic reason it has been set only at the scheduler level other than the AMRM interface change ? We can keep the default value as 1 so that its still compatible. Also anyway allocation happens within app's & queue's capacity limits so i feel it would be ideal for app to decide how many allocations in off_switch node. thoughts ?
          Hide
          leftnoteasy Wangda Tan added a comment -

          Nathan Roberts,

          So maybe there are two things to do:
          1) Have the global OFF_SWITCH check to handle the simple case of avoiding too many network-heavy applications on a node. 
          2) A feature where applications can specify a max_containers_assigned_per_node_per_heartbeat. I think this would be checked down in LeafQueue.assignContainers().
          

          Make sense to me, I agree to limit the scope to OFF_SWITCH allocations in this JIRA.

          Even i feel the same, any specfic reason it has been set only at the scheduler level other than the AMRM interface change ? We can keep the default value as 1 so that its still compatible. Also anyway allocation happens within app's & queue's capacity limits so i feel it would be ideal for app to decide how many allocations in off_switch node. thoughts ?

          Beyond maximum off-switch allocation per node heartbeat, there're some other scheduler global options we may need to consider to move to per-app. One example is locality delays for different locality type.

          Show
          leftnoteasy Wangda Tan added a comment - Nathan Roberts , So maybe there are two things to do : 1) Have the global OFF_SWITCH check to handle the simple case of avoiding too many network-heavy applications on a node. 2) A feature where applications can specify a max_containers_assigned_per_node_per_heartbeat. I think this would be checked down in LeafQueue.assignContainers(). Make sense to me, I agree to limit the scope to OFF_SWITCH allocations in this JIRA. Even i feel the same, any specfic reason it has been set only at the scheduler level other than the AMRM interface change ? We can keep the default value as 1 so that its still compatible. Also anyway allocation happens within app's & queue's capacity limits so i feel it would be ideal for app to decide how many allocations in off_switch node. thoughts ? Beyond maximum off-switch allocation per node heartbeat, there're some other scheduler global options we may need to consider to move to per-app. One example is locality delays for different locality type.
          Hide
          nroberts Nathan Roberts added a comment -

          IMO, I think application specific configurations should be there rather at scheduler level. Some applications are fine with assigning containers in off_switch they can specify number of containers to be assigned. But few applications are very strict to node locality, they can configure 1 in off_switch.

          Even i feel the same, any specfic reason it has been set only at the scheduler level other than the AMRM interface change ? We can keep the default value as 1 so that its still compatible. Also anyway allocation happens within app's & queue's capacity limits so i feel it would be ideal for app to decide how many allocations in off_switch node. thoughts ?

          Thanks Naganarasimha G R, Rohith Sharma K S, Wangda Tan for the comments. I think we're all in agreement that there needs to be some control at the application level for things like OFF_SWITCH allocations, and locality delays (That's what #2 was going for and I think that should be a separate jira if folks are agreeable to that.) This new feature will require some discussion:

          • The current value of 1 is not a good value for almost all applications so I think when we do the application-level support the default would need to be either unlimited or some high value, otherwise we force all applications to set this limit to something other than 1 to get decent OFF_SWITCH scheduling behavior.
          • This setting not only affects the application at hand, but can also affect the entire system. I can see many cases where applications will relax these settings significantly so that their application schedules faster, however that may not have been the right thing for the system as a whole. Sure, my application scheduled very quickly but my locality was terrible so I caused a lot of unnecessary cross-switch traffic. So I think we'll need some system-minimums that will prevent this type of abuse.
          • These changes would potentially affect the fifo-ness of the queues. If application A meets its OFF-SWITCH-per-node limit, do we offer the node to other applications in the same queue?

          So my suggestion is:
          1) Have this jira make the system-level OFF-SWITCH check configurable so admins can easily crank this up and dramatically improve scheduling rate.
          2) Have a second jira to address per-application settings for things like locality_delay and off_switch limits.

          Reasonable?

          Show
          nroberts Nathan Roberts added a comment - IMO, I think application specific configurations should be there rather at scheduler level. Some applications are fine with assigning containers in off_switch they can specify number of containers to be assigned. But few applications are very strict to node locality, they can configure 1 in off_switch. Even i feel the same, any specfic reason it has been set only at the scheduler level other than the AMRM interface change ? We can keep the default value as 1 so that its still compatible. Also anyway allocation happens within app's & queue's capacity limits so i feel it would be ideal for app to decide how many allocations in off_switch node. thoughts ? Thanks Naganarasimha G R , Rohith Sharma K S , Wangda Tan for the comments. I think we're all in agreement that there needs to be some control at the application level for things like OFF_SWITCH allocations, and locality delays (That's what #2 was going for and I think that should be a separate jira if folks are agreeable to that.) This new feature will require some discussion: The current value of 1 is not a good value for almost all applications so I think when we do the application-level support the default would need to be either unlimited or some high value, otherwise we force all applications to set this limit to something other than 1 to get decent OFF_SWITCH scheduling behavior. This setting not only affects the application at hand, but can also affect the entire system. I can see many cases where applications will relax these settings significantly so that their application schedules faster, however that may not have been the right thing for the system as a whole. Sure, my application scheduled very quickly but my locality was terrible so I caused a lot of unnecessary cross-switch traffic. So I think we'll need some system-minimums that will prevent this type of abuse. These changes would potentially affect the fifo-ness of the queues. If application A meets its OFF-SWITCH-per-node limit, do we offer the node to other applications in the same queue? So my suggestion is: 1) Have this jira make the system-level OFF-SWITCH check configurable so admins can easily crank this up and dramatically improve scheduling rate. 2) Have a second jira to address per-application settings for things like locality_delay and off_switch limits. Reasonable?
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Thanks for the clarification Tan, Wangda & Nathan Roberts, yes point 2 addresses the same issue and my mistake i missed to read this. And also agree to the focus of this jira to be specific to the system level OFF-SWITCH configuration.

          so I think when we do the application-level support the default would need to be either unlimited or some high value, otherwise we force all applications to set this limit to something other than 1 to get decent OFF_SWITCH scheduling behavior.

          Once we have system level OFF-SWITCH configuration do we require app level default also ? IIUC by default we try to make use of system level OFF-SWITCH configuration unless explicitly overridden by the app (implementation can be further discussed in that jira)

          Sure, my application scheduled very quickly but my locality was terrible so I caused a lot of unnecessary cross-switch traffic. So I think we'll need some system-minimums that will prevent this type of abuse.

          This point is debatable, even though i agree your point for controlling cross-switch traffic, but still the app is performing under its capacity limits so would it be good to limit it control it.

          If application A meets its OFF-SWITCH-per-node limit, do we offer the node to other applications in the same queue?

          any limitations if we offer the node to other applications in the same queue ? it should be fine right ?

          Show
          Naganarasimha Naganarasimha G R added a comment - Thanks for the clarification Tan, Wangda & Nathan Roberts , yes point 2 addresses the same issue and my mistake i missed to read this. And also agree to the focus of this jira to be specific to the system level OFF-SWITCH configuration. so I think when we do the application-level support the default would need to be either unlimited or some high value, otherwise we force all applications to set this limit to something other than 1 to get decent OFF_SWITCH scheduling behavior. Once we have system level OFF-SWITCH configuration do we require app level default also ? IIUC by default we try to make use of system level OFF-SWITCH configuration unless explicitly overridden by the app (implementation can be further discussed in that jira) Sure, my application scheduled very quickly but my locality was terrible so I caused a lot of unnecessary cross-switch traffic. So I think we'll need some system-minimums that will prevent this type of abuse. This point is debatable, even though i agree your point for controlling cross-switch traffic, but still the app is performing under its capacity limits so would it be good to limit it control it. If application A meets its OFF-SWITCH-per-node limit, do we offer the node to other applications in the same queue? any limitations if we offer the node to other applications in the same queue ? it should be fine right ?
          Hide
          nroberts Nathan Roberts added a comment -

          Sorry it took so long to get back to this. I filed YARN-5013 to handle #2. We can continue that discussion over there.

          Show
          nroberts Nathan Roberts added a comment - Sorry it took so long to get back to this. I filed YARN-5013 to handle #2. We can continue that discussion over there.
          Hide
          nroberts Nathan Roberts added a comment -

          Address checkstyle comment.

          Show
          nroberts Nathan Roberts added a comment - Address checkstyle comment.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 47s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 40s trunk passed
          +1 compile 0m 45s trunk passed with JDK v1.8.0_91
          +1 compile 0m 32s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 21s trunk passed
          +1 mvnsite 0m 41s trunk passed
          +1 mvneclipse 0m 16s trunk passed
          +1 findbugs 1m 20s trunk passed
          +1 javadoc 0m 33s trunk passed with JDK v1.8.0_91
          +1 javadoc 0m 31s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 40s the patch passed
          +1 compile 0m 40s the patch passed with JDK v1.8.0_91
          +1 javac 0m 40s the patch passed
          +1 compile 0m 31s the patch passed with JDK v1.7.0_95
          +1 javac 0m 31s the patch passed
          +1 checkstyle 0m 18s the patch passed
          +1 mvnsite 0m 42s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 1m 34s the patch passed
          +1 javadoc 0m 31s the patch passed with JDK v1.8.0_91
          +1 javadoc 0m 29s the patch passed with JDK v1.7.0_95
          -1 unit 38m 52s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91.
          -1 unit 38m 16s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 20s Patch does not generate ASF License warnings.
          98m 43s



          Reason Tests
          JDK v1.8.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart
            hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
            hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12802502/YARN-4963.002.patch
          JIRA Issue YARN-4963
          Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 28da8887c53c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 35cf503
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11346/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11346/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11346/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/11346/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11346/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11346/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 47s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 40s trunk passed +1 compile 0m 45s trunk passed with JDK v1.8.0_91 +1 compile 0m 32s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 41s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 1m 20s trunk passed +1 javadoc 0m 33s trunk passed with JDK v1.8.0_91 +1 javadoc 0m 31s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 40s the patch passed +1 compile 0m 40s the patch passed with JDK v1.8.0_91 +1 javac 0m 40s the patch passed +1 compile 0m 31s the patch passed with JDK v1.7.0_95 +1 javac 0m 31s the patch passed +1 checkstyle 0m 18s the patch passed +1 mvnsite 0m 42s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 1m 34s the patch passed +1 javadoc 0m 31s the patch passed with JDK v1.8.0_91 +1 javadoc 0m 29s the patch passed with JDK v1.7.0_95 -1 unit 38m 52s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_91. -1 unit 38m 16s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 20s Patch does not generate ASF License warnings. 98m 43s Reason Tests JDK v1.8.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart   hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12802502/YARN-4963.002.patch JIRA Issue YARN-4963 Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 28da8887c53c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 35cf503 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/11346/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11346/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11346/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-YARN-Build/11346/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11346/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/11346/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          nroberts Nathan Roberts added a comment -

          I don't believe test failures are related to this change.

          If someone has a few cycles for a review that would be great. This patch covers step #1 agreed to above. step #2 will be handled via YARN-5013.

          Show
          nroberts Nathan Roberts added a comment - I don't believe test failures are related to this change. If someone has a few cycles for a review that would be great. This patch covers step #1 agreed to above. step #2 will be handled via YARN-5013 .
          Hide
          leftnoteasy Wangda Tan added a comment -

          Thanks Nathan Roberts, few comments:

          1) Configuration name:
          How about call it: yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-assignments?
          offswitch-per-node-limit is a little confusing to me. And in the future we can add other limits under per-node-heartbeat if needed.

          2) We may only need to add getOffSwitchNodeLimit to ParentQueue (instead of adding it to AbstractCSQueue)

          3) (Minor) Logics in ParentQueue:
          Add isDebugEnabled for:

          500             LOG.debug("Not assigning more than " + getOffSwitchNodeLimit() +
          501                 " off-switch containers," +
          

          And it's better to print offswitchCount with the debug log.

          Thoughts?

          Show
          leftnoteasy Wangda Tan added a comment - Thanks Nathan Roberts , few comments: 1) Configuration name: How about call it: yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-assignments? offswitch-per-node-limit is a little confusing to me. And in the future we can add other limits under per-node-heartbeat if needed. 2) We may only need to add getOffSwitchNodeLimit to ParentQueue (instead of adding it to AbstractCSQueue) 3) (Minor) Logics in ParentQueue: Add isDebugEnabled for: 500 LOG.debug( "Not assigning more than " + getOffSwitchNodeLimit() + 501 " off- switch containers," + And it's better to print offswitchCount with the debug log. Thoughts?
          Hide
          nroberts Nathan Roberts added a comment -

          Thank you Wangda Tan for the comments!

          I have addressed them in the latest patch.

          Show
          nroberts Nathan Roberts added a comment - Thank you Wangda Tan for the comments! I have addressed them in the latest patch.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Latest patch looks good, thanks Nathan Roberts!

          Show
          leftnoteasy Wangda Tan added a comment - Latest patch looks good, thanks Nathan Roberts !
          Hide
          jlowe Jason Lowe added a comment -

          The patch no longer applies. Nathan Roberts could you please rebase it?

          Show
          jlowe Jason Lowe added a comment - The patch no longer applies. Nathan Roberts could you please rebase it?
          Hide
          nroberts Nathan Roberts added a comment -

          rebased with trunk

          Show
          nroberts Nathan Roberts added a comment - rebased with trunk
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 2s trunk passed
          +1 compile 0m 36s trunk passed
          +1 checkstyle 0m 22s trunk passed
          +1 mvnsite 0m 39s trunk passed
          +1 mvneclipse 0m 18s trunk passed
          +1 findbugs 0m 55s trunk passed
          +1 javadoc 0m 20s trunk passed
          +1 mvninstall 0m 30s the patch passed
          +1 compile 0m 31s the patch passed
          +1 javac 0m 31s the patch passed
          -0 checkstyle 0m 21s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 4 new + 146 unchanged - 1 fixed = 150 total (was 147)
          +1 mvnsite 0m 36s the patch passed
          +1 mvneclipse 0m 16s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 findbugs 1m 4s the patch passed
          +1 javadoc 0m 18s the patch passed
          +1 unit 35m 19s hadoop-yarn-server-resourcemanager in the patch passed.
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          50m 58s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue YARN-4963
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12835666/YARN-4963.004.patch
          Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 30921385ddd9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / dd4ed6a
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13583/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13583/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/13583/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 2s trunk passed +1 compile 0m 36s trunk passed +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 39s trunk passed +1 mvneclipse 0m 18s trunk passed +1 findbugs 0m 55s trunk passed +1 javadoc 0m 20s trunk passed +1 mvninstall 0m 30s the patch passed +1 compile 0m 31s the patch passed +1 javac 0m 31s the patch passed -0 checkstyle 0m 21s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 4 new + 146 unchanged - 1 fixed = 150 total (was 147) +1 mvnsite 0m 36s the patch passed +1 mvneclipse 0m 16s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 1m 4s the patch passed +1 javadoc 0m 18s the patch passed +1 unit 35m 19s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 50m 58s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue YARN-4963 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12835666/YARN-4963.004.patch Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 30921385ddd9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / dd4ed6a Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13583/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13583/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/13583/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jlowe Jason Lowe added a comment -

          +1 lgtm. Will commit this tomorrow if there are no objections.

          Show
          jlowe Jason Lowe added a comment - +1 lgtm. Will commit this tomorrow if there are no objections.
          Hide
          jlowe Jason Lowe added a comment -

          Thanks to Nathan Roberts for the contribution and to Wangda Tan, Rohith Sharma K S, and Naganarasimha G R for additional review! I committed this to trunk, branch-2, and branch-2.8.

          Show
          jlowe Jason Lowe added a comment - Thanks to Nathan Roberts for the contribution and to Wangda Tan , Rohith Sharma K S , and Naganarasimha G R for additional review! I committed this to trunk, branch-2, and branch-2.8.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10722 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10722/)
          YARN-4963. capacity scheduler: Make number of OFF_SWITCH assignments per (jlowe: rev 1eae719bcead45915977aa220324650eab3c1b9e)

          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10722 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10722/ ) YARN-4963 . capacity scheduler: Make number of OFF_SWITCH assignments per (jlowe: rev 1eae719bcead45915977aa220324650eab3c1b9e) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java

            People

            • Assignee:
              nroberts Nathan Roberts
              Reporter:
              nroberts Nathan Roberts
            • Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development