Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3893

allow capacity scheduler configs maximum-applications and maximum-am-resource-percent configurable on a per queue basis

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: mrv2
    • Labels:
      None

      Description

      The capacity scheduler configs for maximum-applications and maximum-am-resource-percent are currently configured globally and then made proportional to each queue based on its capacity. There are times when this may not work well. some exampless - if you have a queue that is running on uberAM jobs, the jobs a queue is running always has a small number of containers, and then you have the opposite where in a queue with very small capacity, you may want to limit the am resources even more so you don't end up deadlocked with all your capacity being used for app masters.

      I think we should make those configurable on a per queue basis.

      1. MAPREDUCE-3893-1.txt
        9 kB
        Eric Payne
      2. MAPREDUCE-3893.patch
        15 kB
        Thomas Graves

        Activity

        Hide
        Eric Payne added a comment -

        I am marking this as critical because the deadlock situation is a real possibility when one queue is dedicated to one user that submits a large number of jobs simultaneously in a currently quiet cluster. We have a particular use case (GDM) that encounters this situation rebularly.

        Show
        Eric Payne added a comment - I am marking this as critical because the deadlock situation is a real possibility when one queue is dedicated to one user that submits a large number of jobs simultaneously in a currently quiet cluster. We have a particular use case (GDM) that encounters this situation rebularly.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12516901/MAPREDUCE-3893-1.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1990//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1990//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12516901/MAPREDUCE-3893-1.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1990//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1990//console This message is automatically generated.
        Hide
        Arun C Murthy added a comment -

        Eric, I'm not convinced that making it configurable per-queue will solve the deadlock problem. I'm worried making it more complicated will mean more room for error i.e. you could introduce more deadlocks by being over-aggressive with #applications in a queue due to mis-config leading to more deadlocks.

        Could you please share the configs you saw which led to the deadlock? Thanks!

        Show
        Arun C Murthy added a comment - Eric, I'm not convinced that making it configurable per-queue will solve the deadlock problem. I'm worried making it more complicated will mean more room for error i.e. you could introduce more deadlocks by being over-aggressive with #applications in a queue due to mis-config leading to more deadlocks. Could you please share the configs you saw which led to the deadlock? Thanks!
        Hide
        Arun C Murthy added a comment -

        Also, for small clusters we should cut down max-system-apps to 100 or so.

        Show
        Arun C Murthy added a comment - Also, for small clusters we should cut down max-system-apps to 100 or so.
        Hide
        Thomas Graves added a comment -

        I think its more of being able to configure it differently per queue and get better utilization then stopping deadlocks (Although it could also work around deadlock issues). Right now there is only one setting so if you have queues that are running very different jobs, its not very efficient. For instance, what if one queue is running all uberAM's, You want that queue to have maximum-am-resource-percent close to 100%, but if another queue on the same cluster is running jobs with lots of maps/reduces per job you may only want it at 10% because starting other AM's will just waste memory as they will have to wait for the first job anyway. So lets say you set it at 50% to get somewhere in the middle. That means you are wasting half of the queue that wants to run all uberAM's. That 50% may also be to high if you have yet another queue that is set to use 1% of the cluster capacity. Note that the calculation for maxActiveApps is using the minimumAllocation setting, so if a user sets its AM memory to be much more then that and the queue is small, you could deadlock again.

        Note that I'm mostly interested in maximum-am-resource-percent. The maximum-applications we can leave alone.

        I believe the way Eric has it only makes it more complicated if you want to configure it per queue. The global is still there but if you want to override then you can set it per queue. Obviously more settings does introduce the chance for more mis-config but I don't see it anymore complicated then what is already there per queue. I'm worried without this we won't be using the cluster very efficiently or we will run into deadlocks because we set it high but the smaller capacity queues won't be able to handle that.

        If you have other ideas or thoughts please let us know.

        Show
        Thomas Graves added a comment - I think its more of being able to configure it differently per queue and get better utilization then stopping deadlocks (Although it could also work around deadlock issues). Right now there is only one setting so if you have queues that are running very different jobs, its not very efficient. For instance, what if one queue is running all uberAM's, You want that queue to have maximum-am-resource-percent close to 100%, but if another queue on the same cluster is running jobs with lots of maps/reduces per job you may only want it at 10% because starting other AM's will just waste memory as they will have to wait for the first job anyway. So lets say you set it at 50% to get somewhere in the middle. That means you are wasting half of the queue that wants to run all uberAM's. That 50% may also be to high if you have yet another queue that is set to use 1% of the cluster capacity. Note that the calculation for maxActiveApps is using the minimumAllocation setting, so if a user sets its AM memory to be much more then that and the queue is small, you could deadlock again. Note that I'm mostly interested in maximum-am-resource-percent. The maximum-applications we can leave alone. I believe the way Eric has it only makes it more complicated if you want to configure it per queue. The global is still there but if you want to override then you can set it per queue. Obviously more settings does introduce the chance for more mis-config but I don't see it anymore complicated then what is already there per queue. I'm worried without this we won't be using the cluster very efficiently or we will run into deadlocks because we set it high but the smaller capacity queues won't be able to handle that. If you have other ideas or thoughts please let us know.
        Hide
        Thomas Graves added a comment -

        Arun, any input on this?

        On the configs - would it be better to remove the global setting and just have this per queue, that way it would match the other queue configs. I'm not sure if that is considered breaking api compatibility though.

        Show
        Thomas Graves added a comment - Arun, any input on this? On the configs - would it be better to remove the global setting and just have this per queue, that way it would match the other queue configs. I'm not sure if that is considered breaking api compatibility though.
        Hide
        Thomas Graves added a comment -

        patch upmerged to trunk. Fixed a couple issues with maxApplications per queue settings, added documentation, add more tests.

        Show
        Thomas Graves added a comment - patch upmerged to trunk. Fixed a couple issues with maxApplications per queue settings, added documentation, add more tests.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12537364/MAPREDUCE-3893.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        -1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2636//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2636//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2636//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12537364/MAPREDUCE-3893.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2636//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2636//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2636//console This message is automatically generated.
        Hide
        Thomas Graves added a comment -
        Show
        Thomas Graves added a comment - findbugs are unrelated - see https://issues.apache.org/jira/browse/MAPREDUCE-4452
        Hide
        Robert Joseph Evans added a comment -

        The latest set of changes look good to me +1.

        Show
        Robert Joseph Evans added a comment - The latest set of changes look good to me +1.
        Hide
        Robert Joseph Evans added a comment -

        Thanks Tom,

        I put this into trunk, branch-2, and branch-0.23

        Show
        Robert Joseph Evans added a comment - Thanks Tom, I put this into trunk, branch-2, and branch-0.23
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2512 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2512/)
        MAPREDUCE-3893. allow capacity scheduler configs max-apps and max-am-pct per queue (tgraves via bobby) (Revision 1364764)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1364764
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2512 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2512/ ) MAPREDUCE-3893 . allow capacity scheduler configs max-apps and max-am-pct per queue (tgraves via bobby) (Revision 1364764) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1364764 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2577 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2577/)
        MAPREDUCE-3893. allow capacity scheduler configs max-apps and max-am-pct per queue (tgraves via bobby) (Revision 1364764)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1364764
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2577 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2577/ ) MAPREDUCE-3893 . allow capacity scheduler configs max-apps and max-am-pct per queue (tgraves via bobby) (Revision 1364764) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1364764 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2533 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2533/)
        MAPREDUCE-3893. allow capacity scheduler configs max-apps and max-am-pct per queue (tgraves via bobby) (Revision 1364764)

        Result = FAILURE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1364764
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2533 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2533/ ) MAPREDUCE-3893 . allow capacity scheduler configs max-apps and max-am-pct per queue (tgraves via bobby) (Revision 1364764) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1364764 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #323 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/323/)
        svn merge -c 1364764 FIXES: MAPREDUCE-3893. allow capacity scheduler configs max-apps and max-am-pct per queue (tgraves via bobby) (Revision 1364766)

        Result = UNSTABLE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1364766
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #323 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/323/ ) svn merge -c 1364764 FIXES: MAPREDUCE-3893 . allow capacity scheduler configs max-apps and max-am-pct per queue (tgraves via bobby) (Revision 1364766) Result = UNSTABLE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1364766 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1114 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1114/)
        MAPREDUCE-3893. allow capacity scheduler configs max-apps and max-am-pct per queue (tgraves via bobby) (Revision 1364764)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1364764
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1114 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1114/ ) MAPREDUCE-3893 . allow capacity scheduler configs max-apps and max-am-pct per queue (tgraves via bobby) (Revision 1364764) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1364764 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1146 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1146/)
        MAPREDUCE-3893. allow capacity scheduler configs max-apps and max-am-pct per queue (tgraves via bobby) (Revision 1364764)

        Result = FAILURE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1364764
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1146 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1146/ ) MAPREDUCE-3893 . allow capacity scheduler configs max-apps and max-am-pct per queue (tgraves via bobby) (Revision 1364764) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1364764 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/CapacityScheduler.apt.vm

          People

          • Assignee:
            Thomas Graves
            Reporter:
            Thomas Graves
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development