Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3231

FairScheduler: Changing queueMaxRunningApps interferes with pending jobs

    Details

    • Hadoop Flags:
      Reviewed

      Description

      When a queue is piling up with a lot of pending jobs due to the maxRunningApps limit. We want to increase this property on the fly to make some of the pending job active. However, once we increase the limit, all pending jobs were not assigned any resource, and were stuck forever.

      1. YARN-3231.v1.patch
        8 kB
        Siqi Li
      2. YARN-3231.v2.patch
        12 kB
        Siqi Li
      3. YARN-3231.v3.patch
        10 kB
        Siqi Li
      4. YARN-3231.v4.patch
        16 kB
        Siqi Li

        Issue Links

          Activity

          Hide
          l201514 Siqi Li added a comment -

          The problem seems come from MaxRunningAppsEnforcer, I will upload a patch shortly

          Show
          l201514 Siqi Li added a comment - The problem seems come from MaxRunningAppsEnforcer, I will upload a patch shortly
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12699776/YARN-3231.v1.patch
          against trunk revision d49ae72.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6678//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12699776/YARN-3231.v1.patch against trunk revision d49ae72. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6678//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12700280/YARN-3231.v1.patch
          against trunk revision fe7a302.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          -1 javac. The patch appears to cause the build to fail.

          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6703//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12700280/YARN-3231.v1.patch against trunk revision fe7a302. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. -1 javac . The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6703//console This message is automatically generated.
          Hide
          mingma Ming Ma added a comment -

          Siqi, the newly added updateRunnabilityOnRefreshQueues has similar code as updateRunnabilityOnAppRemoval to trim appsNowMaybeRunnable. Will it be useful to put that into a separate function that both updateRunnabilityOnRefreshQueues and updateRunnabilityOnAppRemoval can use? On a separate note, it might be useful to add some test case for userMaxApps change.

          Show
          mingma Ming Ma added a comment - Siqi, the newly added updateRunnabilityOnRefreshQueues has similar code as updateRunnabilityOnAppRemoval to trim appsNowMaybeRunnable . Will it be useful to put that into a separate function that both updateRunnabilityOnRefreshQueues and updateRunnabilityOnAppRemoval can use? On a separate note, it might be useful to add some test case for userMaxApps change.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12700551/YARN-3231.v1.patch
          against trunk revision 73bcfa9.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 5 new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

          org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6711//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/6711//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6711//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12700551/YARN-3231.v1.patch against trunk revision 73bcfa9. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 5 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6711//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/6711//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6711//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12700592/YARN-3231.v2.patch
          against trunk revision 9a37247.

          -1 patch. Trunk compilation may be broken.

          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6714//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12700592/YARN-3231.v2.patch against trunk revision 9a37247. -1 patch . Trunk compilation may be broken. Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6714//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12700597/YARN-3231.v2.patch
          against trunk revision 9a37247.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          -1 javac. The patch appears to cause the build to fail.

          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6715//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12700597/YARN-3231.v2.patch against trunk revision 9a37247. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. -1 javac . The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6715//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12700610/YARN-3231.v2.patch
          against trunk revision 7105eba.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 5 new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

          org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6716//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/6716//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6716//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12700610/YARN-3231.v2.patch against trunk revision 7105eba. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 5 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6716//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/6716//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6716//console This message is automatically generated.
          Hide
          l201514 Siqi Li added a comment -

          find bugs and test failures seems not to be related to this patch

          Show
          l201514 Siqi Li added a comment - find bugs and test failures seems not to be related to this patch
          Hide
          mingma Ming Ma added a comment -

          LGTM.

          Show
          mingma Ming Ma added a comment - LGTM.
          Hide
          kasha Karthik Kambatla added a comment -

          Thanks for reporting and working on this, Siqi Li. The approach looks generally good. Few comments (some nits):

          1. Rename updateRunnabilityonRefreshQueues to updateRunnabilityOnReload? And, add a javadoc for when it should be called and what it does.
          2. javadoc for the newly added private method and the significance of the new integer param.
          3. Call the above method from AllocationReloadListner#onReload after all the other queue configs are updated.
          4. The comment here no longer applies. Remove it?
                    // No more than one app per list will be able to be made runnable, so
                    // we can stop looking after we've found that many
                    if (noLongerPendingApps.size() >= maxRunnableApps) {
                      break;
                    }
            
          5. Indentation:
                updateAppsRunnability(appsNowMaybeRunnable,
            		appsNowMaybeRunnable.size());
            
          6. Newly added tests:
            1. If it is not too much trouble, can we move them to a new test class (TestAppRunnability?) mostly because TestFairScheduler has so many tests already.
            2. Is it possible to reuse the code between these tests?
            3. Should we add tests for when the maxRunnableApps for a user or queue is decreased? If you think this might need additional work in the logic as well, I am open to filing a follow up JIRA and addressing it there.
          Show
          kasha Karthik Kambatla added a comment - Thanks for reporting and working on this, Siqi Li . The approach looks generally good. Few comments (some nits): Rename updateRunnabilityonRefreshQueues to updateRunnabilityOnReload ? And, add a javadoc for when it should be called and what it does. javadoc for the newly added private method and the significance of the new integer param. Call the above method from AllocationReloadListner#onReload after all the other queue configs are updated. The comment here no longer applies. Remove it? // No more than one app per list will be able to be made runnable, so // we can stop looking after we've found that many if (noLongerPendingApps.size() >= maxRunnableApps) { break ; } Indentation: updateAppsRunnability(appsNowMaybeRunnable, appsNowMaybeRunnable.size()); Newly added tests: If it is not too much trouble, can we move them to a new test class (TestAppRunnability?) mostly because TestFairScheduler has so many tests already. Is it possible to reuse the code between these tests? Should we add tests for when the maxRunnableApps for a user or queue is decreased? If you think this might need additional work in the logic as well, I am open to filing a follow up JIRA and addressing it there.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12701160/YARN-3231.v3.patch
          against trunk revision f0c980a.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 5 new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6763//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/6763//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6763//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12701160/YARN-3231.v3.patch against trunk revision f0c980a. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 5 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6763//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/6763//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6763//console This message is automatically generated.
          Hide
          l201514 Siqi Li added a comment -

          Hi Karthik Kambatla, thanks for your feedback.

          I have updated a new patch which addressed all your comment except 6.1 and 6.3.

          For 6.1, it seems that there are other test cases that also might be qualified for moving to TestAppRunnability, it would be good to do a larger refactor of TestFairScheduler into TestAppRunnability.

          For 6.3, I don't think there is a problem with "maxRunnableApps for a user or queue is decreased".

          Show
          l201514 Siqi Li added a comment - Hi Karthik Kambatla , thanks for your feedback. I have updated a new patch which addressed all your comment except 6.1 and 6.3. For 6.1, it seems that there are other test cases that also might be qualified for moving to TestAppRunnability, it would be good to do a larger refactor of TestFairScheduler into TestAppRunnability. For 6.3, I don't think there is a problem with "maxRunnableApps for a user or queue is decreased".
          Hide
          kasha Karthik Kambatla added a comment -

          Filed YARN-3271 to move these tests. I am okay with moving these too as part of that. I will be glad to review that JIRA too, should anyone want to pick it up.

          For 6.3, I don't think there is a problem with "maxRunnableApps for a user or queue is decreased".

          Would be nice to add the tests even if there is no problem. Seems like a logical extension of what the latest patch is doing here.

          Show
          kasha Karthik Kambatla added a comment - Filed YARN-3271 to move these tests. I am okay with moving these too as part of that. I will be glad to review that JIRA too, should anyone want to pick it up. For 6.3, I don't think there is a problem with "maxRunnableApps for a user or queue is decreased". Would be nice to add the tests even if there is no problem. Seems like a logical extension of what the latest patch is doing here.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12701186/YARN-3231.v4.patch
          against trunk revision c6d5b37.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 5 new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6767//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/6767//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6767//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12701186/YARN-3231.v4.patch against trunk revision c6d5b37. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 5 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6767//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/6767//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6767//console This message is automatically generated.
          Hide
          l201514 Siqi Li added a comment -

          Hi Karthik Kambatla, I have added two test cases for reducing User/QueueMaxApps.

          Show
          l201514 Siqi Li added a comment - Hi Karthik Kambatla , I have added two test cases for reducing User/QueueMaxApps.
          Hide
          kasha Karthik Kambatla added a comment -

          LGTM. +1. Checking this in.

          Show
          kasha Karthik Kambatla added a comment - LGTM. +1. Checking this in.
          Hide
          kasha Karthik Kambatla added a comment -

          Thanks Siqi Li for fixing this.

          Just committed this to trunk and branch-2.

          Show
          kasha Karthik Kambatla added a comment - Thanks Siqi Li for fixing this. Just committed this to trunk and branch-2.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7261 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7261/)
          YARN-3231. FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi Li via kasha) (kasha: rev 22426a1c9f4bd616558089b6862fd34ab42d19a7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/MaxRunningAppsEnforcer.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7261 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7261/ ) YARN-3231 . FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi Li via kasha) (kasha: rev 22426a1c9f4bd616558089b6862fd34ab42d19a7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/MaxRunningAppsEnforcer.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #123 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/123/)
          YARN-3231. FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi Li via kasha) (kasha: rev 22426a1c9f4bd616558089b6862fd34ab42d19a7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/MaxRunningAppsEnforcer.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #123 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/123/ ) YARN-3231 . FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi Li via kasha) (kasha: rev 22426a1c9f4bd616558089b6862fd34ab42d19a7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/MaxRunningAppsEnforcer.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #857 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/857/)
          YARN-3231. FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi Li via kasha) (kasha: rev 22426a1c9f4bd616558089b6862fd34ab42d19a7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/MaxRunningAppsEnforcer.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #857 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/857/ ) YARN-3231 . FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi Li via kasha) (kasha: rev 22426a1c9f4bd616558089b6862fd34ab42d19a7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/MaxRunningAppsEnforcer.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2055 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2055/)
          YARN-3231. FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi Li via kasha) (kasha: rev 22426a1c9f4bd616558089b6862fd34ab42d19a7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/MaxRunningAppsEnforcer.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2055 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2055/ ) YARN-3231 . FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi Li via kasha) (kasha: rev 22426a1c9f4bd616558089b6862fd34ab42d19a7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/MaxRunningAppsEnforcer.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #114 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/114/)
          YARN-3231. FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi Li via kasha) (kasha: rev 22426a1c9f4bd616558089b6862fd34ab42d19a7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/MaxRunningAppsEnforcer.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #114 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/114/ ) YARN-3231 . FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi Li via kasha) (kasha: rev 22426a1c9f4bd616558089b6862fd34ab42d19a7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/MaxRunningAppsEnforcer.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #123 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/123/)
          YARN-3231. FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi Li via kasha) (kasha: rev 22426a1c9f4bd616558089b6862fd34ab42d19a7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/MaxRunningAppsEnforcer.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #123 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/123/ ) YARN-3231 . FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi Li via kasha) (kasha: rev 22426a1c9f4bd616558089b6862fd34ab42d19a7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/MaxRunningAppsEnforcer.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java hadoop-yarn-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2073 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2073/)
          YARN-3231. FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi Li via kasha) (kasha: rev 22426a1c9f4bd616558089b6862fd34ab42d19a7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/MaxRunningAppsEnforcer.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2073 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2073/ ) YARN-3231 . FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi Li via kasha) (kasha: rev 22426a1c9f4bd616558089b6862fd34ab42d19a7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/MaxRunningAppsEnforcer.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java hadoop-yarn-project/CHANGES.txt
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Pulled this into 2.6.1. Ran compilation and TestFairScheduler before the push. Patch applied cleanly.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Pulled this into 2.6.1. Ran compilation and TestFairScheduler before the push. Patch applied cleanly.

            People

            • Assignee:
              l201514 Siqi Li
              Reporter:
              l201514 Siqi Li
            • Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development