Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3897

capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.2
    • Component/s: mrv2
    • Labels:
      None

      Description

      The capacity scheduler calculates the maxActiveApplications and the maxActiveApplicationsPerUser based on the config yarn.scheduler.capacity.maximum-applications or default 10000.

      MaxActiveApplications = max ( ceil ( clusterMemory/minAllocation * maxAMResource% * absoluteMaxCapacity), 1)

      MaxActiveAppsPerUser = max( ceil (maxActiveApplicationsComputedAbove * (userLimit%/100) * userLimitFactor), 1)

      maxActiveApplications is already multiplied by the queue absolute MAXIMUM capacity, so if max capacity > capacity and if you have user limit factor 1 (which is the default) and only 1 user is running, that user will not be allowed to use over the queue capacity, so having it relative to MAX capacity doesn't make sense. That user could easily end up in a deadlock and all its space used by application masters.

      1. MAPREDUCE-3897-1.txt
        3 kB
        Eric Payne
      2. MAPREDUCE-3897-1.txt
        3 kB
        Eric Payne
      3. MAPREDUCE-3897-2.txt
        5 kB
        Eric Payne

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1010 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1010/)
        MAPREDUCE-3897. Fixed computation of maxActiveAppsPerUser for queues by using capacity and not max-capacity since we are already scaling it by userLimitFactor. Contributed by Eric Payne. (Revision 1296898)

        Result = FAILURE
        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296898
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1010 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1010/ ) MAPREDUCE-3897 . Fixed computation of maxActiveAppsPerUser for queues by using capacity and not max-capacity since we are already scaling it by userLimitFactor. Contributed by Eric Payne. (Revision 1296898) Result = FAILURE acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296898 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Build #216 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/216/)
        Merge -c 1296898 from trunk to branch-0.23 to fix MAPREDUCE-3897. Fixed computation of maxActiveAppsPerUser for queues by using capacity and not max-capacity since we are already scaling it by userLimitFactor. (Revision 1296899)

        Result = FAILURE
        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296899
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #216 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/216/ ) Merge -c 1296898 from trunk to branch-0.23 to fix MAPREDUCE-3897 . Fixed computation of maxActiveAppsPerUser for queues by using capacity and not max-capacity since we are already scaling it by userLimitFactor. (Revision 1296899) Result = FAILURE acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296899 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #188 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/188/)
        Merge -c 1296898 from trunk to branch-0.23 to fix MAPREDUCE-3897. Fixed computation of maxActiveAppsPerUser for queues by using capacity and not max-capacity since we are already scaling it by userLimitFactor. (Revision 1296899)

        Result = SUCCESS
        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296899
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #188 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/188/ ) Merge -c 1296898 from trunk to branch-0.23 to fix MAPREDUCE-3897 . Fixed computation of maxActiveAppsPerUser for queues by using capacity and not max-capacity since we are already scaling it by userLimitFactor. (Revision 1296899) Result = SUCCESS acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296899 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #975 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/975/)
        MAPREDUCE-3897. Fixed computation of maxActiveAppsPerUser for queues by using capacity and not max-capacity since we are already scaling it by userLimitFactor. Contributed by Eric Payne. (Revision 1296898)

        Result = SUCCESS
        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296898
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #975 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/975/ ) MAPREDUCE-3897 . Fixed computation of maxActiveAppsPerUser for queues by using capacity and not max-capacity since we are already scaling it by userLimitFactor. Contributed by Eric Payne. (Revision 1296898) Result = SUCCESS acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296898 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #1838 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1838/)
        MAPREDUCE-3897. Fixed computation of maxActiveAppsPerUser for queues by using capacity and not max-capacity since we are already scaling it by userLimitFactor. Contributed by Eric Payne. (Revision 1296898)

        Result = ABORTED
        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296898
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1838 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1838/ ) MAPREDUCE-3897 . Fixed computation of maxActiveAppsPerUser for queues by using capacity and not max-capacity since we are already scaling it by userLimitFactor. Contributed by Eric Payne. (Revision 1296898) Result = ABORTED acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296898 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Commit #636 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/636/)
        Merge -c 1296898 from trunk to branch-0.23 to fix MAPREDUCE-3897. Fixed computation of maxActiveAppsPerUser for queues by using capacity and not max-capacity since we are already scaling it by userLimitFactor. (Revision 1296899)

        Result = FAILURE
        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296899
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #636 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/636/ ) Merge -c 1296898 from trunk to branch-0.23 to fix MAPREDUCE-3897 . Fixed computation of maxActiveAppsPerUser for queues by using capacity and not max-capacity since we are already scaling it by userLimitFactor. (Revision 1296899) Result = FAILURE acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296899 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-0.23-Commit #637 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/637/)
        Merge -c 1296898 from trunk to branch-0.23 to fix MAPREDUCE-3897. Fixed computation of maxActiveAppsPerUser for queues by using capacity and not max-capacity since we are already scaling it by userLimitFactor. (Revision 1296899)

        Result = SUCCESS
        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296899
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #637 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/637/ ) Merge -c 1296898 from trunk to branch-0.23 to fix MAPREDUCE-3897 . Fixed computation of maxActiveAppsPerUser for queues by using capacity and not max-capacity since we are already scaling it by userLimitFactor. (Revision 1296899) Result = SUCCESS acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296899 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #1831 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1831/)
        MAPREDUCE-3897. Fixed computation of maxActiveAppsPerUser for queues by using capacity and not max-capacity since we are already scaling it by userLimitFactor. Contributed by Eric Payne. (Revision 1296898)

        Result = SUCCESS
        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296898
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1831 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1831/ ) MAPREDUCE-3897 . Fixed computation of maxActiveAppsPerUser for queues by using capacity and not max-capacity since we are already scaling it by userLimitFactor. Contributed by Eric Payne. (Revision 1296898) Result = SUCCESS acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296898 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #1905 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1905/)
        MAPREDUCE-3897. Fixed computation of maxActiveAppsPerUser for queues by using capacity and not max-capacity since we are already scaling it by userLimitFactor. Contributed by Eric Payne. (Revision 1296898)

        Result = SUCCESS
        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296898
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1905 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1905/ ) MAPREDUCE-3897 . Fixed computation of maxActiveAppsPerUser for queues by using capacity and not max-capacity since we are already scaling it by userLimitFactor. Contributed by Eric Payne. (Revision 1296898) Result = SUCCESS acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296898 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Commit #626 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/626/)
        Merge -c 1296898 from trunk to branch-0.23 to fix MAPREDUCE-3897. Fixed computation of maxActiveAppsPerUser for queues by using capacity and not max-capacity since we are already scaling it by userLimitFactor. (Revision 1296899)

        Result = SUCCESS
        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296899
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #626 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/626/ ) Merge -c 1296898 from trunk to branch-0.23 to fix MAPREDUCE-3897 . Fixed computation of maxActiveAppsPerUser for queues by using capacity and not max-capacity since we are already scaling it by userLimitFactor. (Revision 1296899) Result = SUCCESS acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296899 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java
        Arun C Murthy made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 0.23.2 [ 12319851 ]
        Resolution Fixed [ 1 ]
        Hide
        Arun C Murthy added a comment -

        I just committed this. Thanks Eric!

        Show
        Arun C Murthy added a comment - I just committed this. Thanks Eric!
        Hide
        Thomas Graves added a comment -

        I should add, my opinion is the other issue with user minimum limit % can be fixed in a separate jira.

        Show
        Thomas Graves added a comment - I should add, my opinion is the other issue with user minimum limit % can be fixed in a separate jira.
        Hide
        Thomas Graves added a comment -

        +1 for the formula change to fix the current issue described here. I haven't had a chance to review the code in detail. I think there is still a bug in the computation in that it uses the user minimum limit % in the calculation of maxActiveApplicationPerUser, when really that need to factor in the number active users. so similar to computeUserLimit function where it takes the max of userlimit*capacity or capacity/activeusers.

        Perhaps Arun can comment.

        Show
        Thomas Graves added a comment - +1 for the formula change to fix the current issue described here. I haven't had a chance to review the code in detail. I think there is still a bug in the computation in that it uses the user minimum limit % in the calculation of maxActiveApplicationPerUser, when really that need to factor in the number active users. so similar to computeUserLimit function where it takes the max of userlimit*capacity or capacity/activeusers. Perhaps Arun can comment.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12516874/MAPREDUCE-3897-2.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1986//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1986//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12516874/MAPREDUCE-3897-2.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1986//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1986//console This message is automatically generated.
        Eric Payne made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Eric Payne made changes -
        Attachment MAPREDUCE-3897-2.txt [ 12516874 ]
        Hide
        Eric Payne added a comment -

        Updated patch so that maxActiveApplications remains calculated based on absoluteMaxCapacity but modified so that only maxActiveApplicationsPerUser is calculated based on absoluteCapacity.

        This will limit the number of active apps for a single user to a percetage corresponding the absolute capacity of the queue unless the user limit factor is increased for that queue.

        Show
        Eric Payne added a comment - Updated patch so that maxActiveApplications remains calculated based on absoluteMaxCapacity but modified so that only maxActiveApplicationsPerUser is calculated based on absoluteCapacity. This will limit the number of active apps for a single user to a percetage corresponding the absolute capacity of the queue unless the user limit factor is increased for that queue.
        Eric Payne made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Thomas Graves added a comment -

        I just thought of a case where this won't work well for utilization. That is if you have a queue with small capacity - say 1%, but its max capacity is say 100%, even if we had the configuration per queue for am% and you set it really high, it might only be allowed a couple of AM's when in reality if the cluster has no one else running it should be allowed more so it could use the 100% max capacity.

        We might be better off leaving the maxActiveApplications computation using maxCapacity but changing the maxActiveApplicationsPerUser to use capacity and then allow the user limit factor to apply. Need to think about it some more.

        Show
        Thomas Graves added a comment - I just thought of a case where this won't work well for utilization. That is if you have a queue with small capacity - say 1%, but its max capacity is say 100%, even if we had the configuration per queue for am% and you set it really high, it might only be allowed a couple of AM's when in reality if the cluster has no one else running it should be allowed more so it could use the 100% max capacity. We might be better off leaving the maxActiveApplications computation using maxCapacity but changing the maxActiveApplicationsPerUser to use capacity and then allow the user limit factor to apply. Need to think about it some more.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12516619/MAPREDUCE-3897-1.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1973//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1973//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12516619/MAPREDUCE-3897-1.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1973//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1973//console This message is automatically generated.
        Eric Payne made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Eric Payne made changes -
        Attachment MAPREDUCE-3897-1.txt [ 12516619 ]
        Eric Payne made changes -
        Attachment MAPREDUCE-3897-1.txt [ 12516616 ]
        Hide
        Eric Payne added a comment -

        Changed calculation for maxActiveApplications.

        Old: maxActiveApplications = Max((((total cluster mem) / minimumAllocation) * maxAMResourcePercent * absoluteMaxCapacity),1)

        New: maxActiveApplications = Max((((total cluster mem) / minimumAllocation) * maxAMResourcePercent * absoluteCapacity),1)

        By using absoluteCapacity rather than absoluteMaxCapacity, the result is based on a per-queue capacity and prevents a small queue from hitting the deadlock condition of being filled with all AMs.

        This fix will be further enhanced when it is possible to specify maxAMResourcePercent on a per-queue basis (see MAPREDUCE:3893)

        Show
        Eric Payne added a comment - Changed calculation for maxActiveApplications. Old: maxActiveApplications = Max((((total cluster mem) / minimumAllocation) * maxAMResourcePercent * absoluteMaxCapacity),1) New: maxActiveApplications = Max((((total cluster mem) / minimumAllocation) * maxAMResourcePercent * absoluteCapacity),1) By using absoluteCapacity rather than absoluteMaxCapacity, the result is based on a per-queue capacity and prevents a small queue from hitting the deadlock condition of being filled with all AMs. This fix will be further enhanced when it is possible to specify maxAMResourcePercent on a per-queue basis (see MAPREDUCE:3893)
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4112/
        -----------------------------------------------------------

        Review request for Tom Graves.

        Summary
        -------

        MAPREDUCE-3897: capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong

        This addresses bug MAPREDUCE-3897.
        https://issues.apache.org/jira/browse/MAPREDUCE-3897

        Diffs


        branches/branch-0.23.2/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java 1295253
        branches/branch-0.23.2/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java 1295253
        branches/branch-0.23.2/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java 1295253

        Diff: https://reviews.apache.org/r/4112/diff

        Testing
        -------

        Thanks,

        Eric

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4112/ ----------------------------------------------------------- Review request for Tom Graves. Summary ------- MAPREDUCE-3897 : capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong This addresses bug MAPREDUCE-3897 . https://issues.apache.org/jira/browse/MAPREDUCE-3897 Diffs branches/branch-0.23.2/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java 1295253 branches/branch-0.23.2/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java 1295253 branches/branch-0.23.2/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java 1295253 Diff: https://reviews.apache.org/r/4112/diff Testing ------- Thanks, Eric
        Eric Payne made changes -
        Field Original Value New Value
        Assignee Eric Payne [ eepayne ]
        Thomas Graves created issue -

          People

          • Assignee:
            Eric Payne
            Reporter:
            Thomas Graves
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development