Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4066

Large number of queues choke fair scheduler

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: fairscheduler
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Flags:
      Patch

      Description

      Due to synchronization and all the loops performed during queue creation, setting a large amount of queues (12000+) will completely choke the scheduler. To deal with this some optimization to "QueueManager.updateAllocationConfiguration(AllocationConfiguration queueConf)" should be done to reduce the amount of unnesecary loops. The attached patch have been tested to work with atleast 96000 queues.

      1. yarn-4066-1.patch
        4 kB
        Johan Gustavsson
      2. YARN-4066-2.patch
        4 kB
        Johan Gustavsson
      3. YARN-4066-3.patch
        5 kB
        Johan Gustavsson

        Activity

        Hide
        johang Johan Gustavsson added a comment -

        Adding patch

        Show
        johang Johan Gustavsson added a comment - Adding patch
        Hide
        asuresh Arun Suresh added a comment -

        Johan Gustavsson, Thanks for reporting this... I don't think the patch has been successfully uploaded though. Kindly re-attach.

        Show
        asuresh Arun Suresh added a comment - Johan Gustavsson , Thanks for reporting this... I don't think the patch has been successfully uploaded though. Kindly re-attach.
        Hide
        johang Johan Gustavsson added a comment -

        Sorry, I re-uploaded it and I think it should be visible now

        Show
        johang Johan Gustavsson added a comment - Sorry, I re-uploaded it and I think it should be visible now
        Hide
        kasha Karthik Kambatla added a comment -

        The patch looks reasonable. Could you comment on your queue setup (depth, average breadth etc.) for the 96,000 queues that you tested this on? Just curious.

        Show
        kasha Karthik Kambatla added a comment - The patch looks reasonable. Could you comment on your queue setup (depth, average breadth etc.) for the 96,000 queues that you tested this on? Just curious.
        Hide
        asuresh Arun Suresh added a comment -

        +1 pending jenkins

        Show
        asuresh Arun Suresh added a comment - +1 pending jenkins
        Hide
        johang Johan Gustavsson added a comment -

        Basically its a tree as follows ranging from 1 to 16000. For each user group there is one general queue and one with weight divided sub queues.

        root

        • 1
        • q1
        • veryhigh
        • high
        • default
        • low
        • verylow
        Show
        johang Johan Gustavsson added a comment - Basically its a tree as follows ranging from 1 to 16000. For each user group there is one general queue and one with weight divided sub queues. root 1 q1 veryhigh high default low verylow
        Hide
        johang Johan Gustavsson added a comment -

        As I done seem to be able to edit the above comment and the tree ended up weird I'll repast it below

        root:
        1
        q1:
        veryhigh
        high
        default
        low
        verylow

        Show
        johang Johan Gustavsson added a comment - As I done seem to be able to edit the above comment and the tree ended up weird I'll repast it below root: 1 q1: veryhigh high default low verylow
        Hide
        johang Johan Gustavsson added a comment -

        Please let me know if there is anything I can do to continue the process of merging this

        Show
        johang Johan Gustavsson added a comment - Please let me know if there is anything I can do to continue the process of merging this
        Hide
        kasha Karthik Kambatla added a comment -

        Looking into it.

        Show
        kasha Karthik Kambatla added a comment - Looking into it.
        Hide
        kasha Karthik Kambatla added a comment -

        Thanks again for working on this, Johan. Took a closer look at the patch and have the following comments:

        1. A few lines are longer than 80 characters.
        2. For the method parameters, recomputeSteadyShares might be more descriptive thaan recalculate
        3. While at it, I would suggest the following improvements in synchronization as well:
          1. In getQueue, some of the code could be outside the synchronized block
                name = ensureRootPrefix(name);
                FSQueue queue;
                synchronized (queues) {
                  queue = queues.get(name);
                  if (queue == null && create) {
                    // if the queue doesn't exist,create it and return
                    queue = createQueue(name, queueType);
                  } else {
                    recalculate = false;
                  }
                }
            
                if (recalculate) {
                  rootQueue.recomputeSteadyShares();
                }
                return queue;
            
          2. In updateAllocationConfiguration, club the two synchronized blocks into one, and recomputeSteadyShares outside the synchronized block.

        Since we are changing some of the locking that would be hard to unit-tests, would appreciate if you could run the updated patch through the tests you previously reported.

        Show
        kasha Karthik Kambatla added a comment - Thanks again for working on this, Johan. Took a closer look at the patch and have the following comments: A few lines are longer than 80 characters. For the method parameters, recomputeSteadyShares might be more descriptive thaan recalculate While at it, I would suggest the following improvements in synchronization as well: In getQueue, some of the code could be outside the synchronized block name = ensureRootPrefix(name); FSQueue queue; synchronized (queues) { queue = queues.get(name); if (queue == null && create) { // if the queue doesn't exist,create it and return queue = createQueue(name, queueType); } else { recalculate = false ; } } if (recalculate) { rootQueue.recomputeSteadyShares(); } return queue; In updateAllocationConfiguration, club the two synchronized blocks into one, and recomputeSteadyShares outside the synchronized block. Since we are changing some of the locking that would be hard to unit-tests, would appreciate if you could run the updated patch through the tests you previously reported.
        Hide
        johang Johan Gustavsson added a comment -

        Thanks for the feedback Karthik, I'll try to have a fixed patch ready by next week.

        Show
        johang Johan Gustavsson added a comment - Thanks for the feedback Karthik, I'll try to have a fixed patch ready by next week.
        Hide
        johang Johan Gustavsson added a comment -

        I fixed the patch according to the feedback from Karthik Kambatla.
        I also tested it in our test env and check compatibility of the following unit tests.

        • org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestQueueManager
        • org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler
        • org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestMaxRunningAppsEnforcer
        • org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.TestFairSchedulerQueueInfo
        • org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler
        • org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSLeafQueue
        • org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestAppRunnability
        Show
        johang Johan Gustavsson added a comment - I fixed the patch according to the feedback from Karthik Kambatla . I also tested it in our test env and check compatibility of the following unit tests. org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestQueueManager org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestMaxRunningAppsEnforcer org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.TestFairSchedulerQueueInfo org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSLeafQueue org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestAppRunnability
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 16m 42s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 50s There were no new javac warning messages.
        +1 javadoc 10m 2s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 0m 50s The applied patch generated 4 new checkstyle issues (total was 15, now 18).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 30s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 1m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 yarn tests 60m 3s Tests passed in hadoop-yarn-server-resourcemanager.
            99m 25s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12762628/YARN-4066-2.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 1c030c6
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9279/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
        hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9279/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9279/testReport/
        Java 1.7.0_55
        uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/9279/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 42s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 50s There were no new javac warning messages. +1 javadoc 10m 2s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 50s The applied patch generated 4 new checkstyle issues (total was 15, now 18). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 30s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 60m 3s Tests passed in hadoop-yarn-server-resourcemanager.     99m 25s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12762628/YARN-4066-2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 1c030c6 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9279/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9279/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9279/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9279/console This message was automatically generated.
        Hide
        johang Johan Gustavsson added a comment -

        The formatting issues have been taken care of, sorry about that.

        Show
        johang Johan Gustavsson added a comment - The formatting issues have been taken care of, sorry about that.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 16m 34s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 47s There were no new javac warning messages.
        +1 javadoc 9m 59s There were no new javadoc warning messages.
        +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 0m 49s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 29s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 1m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 yarn tests 60m 4s Tests passed in hadoop-yarn-server-resourcemanager.
            99m 11s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12762639/YARN-4066-3.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 1c030c6
        hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9280/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9280/testReport/
        Java 1.7.0_55
        uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/9280/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 34s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 47s There were no new javac warning messages. +1 javadoc 9m 59s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 49s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 29s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 1m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 yarn tests 60m 4s Tests passed in hadoop-yarn-server-resourcemanager.     99m 11s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12762639/YARN-4066-3.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 1c030c6 hadoop-yarn-server-resourcemanager test log https://builds.apache.org/job/PreCommit-YARN-Build/9280/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9280/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-YARN-Build/9280/console This message was automatically generated.
        Hide
        johang Johan Gustavsson added a comment -

        Thanks for re-running this. Looking at the tests in place I see no need to introduce new tests for this patch, also this patch doesn't break the tests in place.

        Show
        johang Johan Gustavsson added a comment - Thanks for re-running this. Looking at the tests in place I see no need to introduce new tests for this patch, also this patch doesn't break the tests in place.
        Hide
        kasha Karthik Kambatla added a comment -

        +1, checking this in.

        Show
        kasha Karthik Kambatla added a comment - +1, checking this in.
        Hide
        kasha Karthik Kambatla added a comment -

        Just committed to trunk and branch-2. Thanks Johan Gustavsson

        Show
        kasha Karthik Kambatla added a comment - Just committed to trunk and branch-2. Thanks Johan Gustavsson
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8540 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8540/)
        YARN-4066. Large number of queues choke fair scheduler. (Johan Gustavsson via kasha) (kasha: rev a0b5a0a419dfc07b7ac45c06b11b4c8dc7e79958)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8540 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8540/ ) YARN-4066 . Large number of queues choke fair scheduler. (Johan Gustavsson via kasha) (kasha: rev a0b5a0a419dfc07b7ac45c06b11b4c8dc7e79958) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #459 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/459/)
        YARN-4066. Large number of queues choke fair scheduler. (Johan Gustavsson via kasha) (kasha: rev a0b5a0a419dfc07b7ac45c06b11b4c8dc7e79958)

        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #459 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/459/ ) YARN-4066 . Large number of queues choke fair scheduler. (Johan Gustavsson via kasha) (kasha: rev a0b5a0a419dfc07b7ac45c06b11b4c8dc7e79958) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #467 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/467/)
        YARN-4066. Large number of queues choke fair scheduler. (Johan Gustavsson via kasha) (kasha: rev a0b5a0a419dfc07b7ac45c06b11b4c8dc7e79958)

        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #467 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/467/ ) YARN-4066 . Large number of queues choke fair scheduler. (Johan Gustavsson via kasha) (kasha: rev a0b5a0a419dfc07b7ac45c06b11b4c8dc7e79958) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #1198 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1198/)
        YARN-4066. Large number of queues choke fair scheduler. (Johan Gustavsson via kasha) (kasha: rev a0b5a0a419dfc07b7ac45c06b11b4c8dc7e79958)

        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1198 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1198/ ) YARN-4066 . Large number of queues choke fair scheduler. (Johan Gustavsson via kasha) (kasha: rev a0b5a0a419dfc07b7ac45c06b11b4c8dc7e79958) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2403 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2403/)
        YARN-4066. Large number of queues choke fair scheduler. (Johan Gustavsson via kasha) (kasha: rev a0b5a0a419dfc07b7ac45c06b11b4c8dc7e79958)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2403 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2403/ ) YARN-4066 . Large number of queues choke fair scheduler. (Johan Gustavsson via kasha) (kasha: rev a0b5a0a419dfc07b7ac45c06b11b4c8dc7e79958) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2375 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2375/)
        YARN-4066. Large number of queues choke fair scheduler. (Johan Gustavsson via kasha) (kasha: rev a0b5a0a419dfc07b7ac45c06b11b4c8dc7e79958)

        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2375 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2375/ ) YARN-4066 . Large number of queues choke fair scheduler. (Johan Gustavsson via kasha) (kasha: rev a0b5a0a419dfc07b7ac45c06b11b4c8dc7e79958) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #435 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/435/)
        YARN-4066. Large number of queues choke fair scheduler. (Johan Gustavsson via kasha) (kasha: rev a0b5a0a419dfc07b7ac45c06b11b4c8dc7e79958)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #435 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/435/ ) YARN-4066 . Large number of queues choke fair scheduler. (Johan Gustavsson via kasha) (kasha: rev a0b5a0a419dfc07b7ac45c06b11b4c8dc7e79958) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java hadoop-yarn-project/CHANGES.txt

          People

          • Assignee:
            johang Johan Gustavsson
            Reporter:
            johang Johan Gustavsson
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development