Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Recently saw a significant slowdown of applications on a large cluster, and we noticed there were a large number of blocked threads on the RM. Most of the blocked threads were waiting for the CapacityScheduler lock while calling getQueueInfo.

      1. YARN-3487.001.patch
        1 kB
        Jason Lowe
      2. YARN-3487.002.patch
        2 kB
        Jason Lowe
      3. YARN-3487.003.patch
        1 kB
        Jason Lowe

        Activity

        Hide
        jlowe Jason Lowe added a comment -

        Sample stacktrace of a blocked thread:

        "IPC Server handler 48 on x" daemon prio=10 tid=0x00007fd4991d5000 nid=0x5d53 waiting for monitor entry [0x00007fd45cf1a000]
           java.lang.Thread.State: BLOCKED (on object monitor)
                at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.getQueueInfo(CapacityScheduler.java:909)
                - waiting to lock <0x000000023ae2c938> (a org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler)
                at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.validateResourceRequest(SchedulerUtils.java:223)
                at org.apache.hadoop.yarn.server.resourcemanager.RMServerUtils.validateResourceRequests(RMServerUtils.java:96)
                at org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:501)
                - locked <0x00000002616389e0> (a org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService$AllocateResponseLock)
                at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationMasterProtocolPBServiceImpl.allocate(ApplicationMasterProtocolPBServiceImpl.java:60)
                at org.apache.hadoop.yarn.proto.ApplicationMasterProtocol$ApplicationMasterProtocolService$2.callBlockingMethod(ApplicationMasterProtocol.java:99)
                at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
                at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
                at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)
                at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)
                at java.security.AccessController.doPrivileged(Native Method)
                at javax.security.auth.Subject.doAs(Subject.java:415)
                at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1694)
                at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)
        

        In 2.6 we started calling getQueueInfo to validate resource requests so we could check the labels. Unfortunately getQueueInfo grabs the scheduler lock unnecessarily:

          public QueueInfo getQueueInfo(String queueName, 
              boolean includeChildQueues, boolean recursive) 
          throws IOException {
            CSQueue queue = null;
        
            synchronized (this) {
              queue = this.queues.get(queueName); 
            }
        
            if (queue == null) {
              throw new IOException("Unknown queue: " + queueName);
            }
            return queue.getQueueInfo(includeChildQueues, recursive);
          }
        

        this.queues is a ConcurrentHashMap, so there's not much utility in grabbing the lock just to lookup something in that map. Worse, it adds a lot of contention on an already highly contentious lock.

        Show
        jlowe Jason Lowe added a comment - Sample stacktrace of a blocked thread: "IPC Server handler 48 on x" daemon prio=10 tid=0x00007fd4991d5000 nid=0x5d53 waiting for monitor entry [0x00007fd45cf1a000] java.lang. Thread .State: BLOCKED (on object monitor) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.getQueueInfo(CapacityScheduler.java:909) - waiting to lock <0x000000023ae2c938> (a org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.validateResourceRequest(SchedulerUtils.java:223) at org.apache.hadoop.yarn.server.resourcemanager.RMServerUtils.validateResourceRequests(RMServerUtils.java:96) at org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:501) - locked <0x00000002616389e0> (a org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService$AllocateResponseLock) at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationMasterProtocolPBServiceImpl.allocate(ApplicationMasterProtocolPBServiceImpl.java:60) at org.apache.hadoop.yarn.proto.ApplicationMasterProtocol$ApplicationMasterProtocolService$2.callBlockingMethod(ApplicationMasterProtocol.java:99) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1694) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080) In 2.6 we started calling getQueueInfo to validate resource requests so we could check the labels. Unfortunately getQueueInfo grabs the scheduler lock unnecessarily: public QueueInfo getQueueInfo( String queueName, boolean includeChildQueues, boolean recursive) throws IOException { CSQueue queue = null ; synchronized ( this ) { queue = this .queues.get(queueName); } if (queue == null ) { throw new IOException( "Unknown queue: " + queueName); } return queue.getQueueInfo(includeChildQueues, recursive); } this.queues is a ConcurrentHashMap, so there's not much utility in grabbing the lock just to lookup something in that map. Worse, it adds a lot of contention on an already highly contentious lock.
        Hide
        jlowe Jason Lowe added a comment -

        Patch that removes the synchronization around the ConcurrentHashMap access.

        Show
        jlowe Jason Lowe added a comment - Patch that removes the synchronization around the ConcurrentHashMap access.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Patch LGTM +1.

        Show
        leftnoteasy Wangda Tan added a comment - Patch LGTM +1.
        Hide
        leftnoteasy Wangda Tan added a comment -

        And we saw such case before, when a remote process calling getQueueInfo very frequently (e.g. once every 1ms), CS cannot allocate anything.

        Show
        leftnoteasy Wangda Tan added a comment - And we saw such case before, when a remote process calling getQueueInfo very frequently (e.g. once every 1ms), CS cannot allocate anything.
        Hide
        jlowe Jason Lowe added a comment -

        Actually there are more locks we should clean up, posting an update shortly.

        Show
        jlowe Jason Lowe added a comment - Actually there are more locks we should clean up, posting an update shortly.
        Hide
        jlowe Jason Lowe added a comment -

        We're also grabbing the scheduler lock in getQueue which just looks up the queue from the concurrent hash map. In addition checkAccess is locking to lookup the queue then invoke a method on the queue.

        Updated the patch to remove the lock on those two methods as well.

        Show
        jlowe Jason Lowe added a comment - We're also grabbing the scheduler lock in getQueue which just looks up the queue from the concurrent hash map. In addition checkAccess is locking to lookup the queue then invoke a method on the queue. Updated the patch to remove the lock on those two methods as well.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12725395/YARN-3487.001.patch
        against trunk revision 05007b4.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

        Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7334//testReport/
        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7334//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12725395/YARN-3487.001.patch against trunk revision 05007b4. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7334//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7334//console This message is automatically generated.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Jason Lowe, removing lock of checkAccess may cause consumer get partial queue's ACL when reinitialize happens. See YarnAuthorizationProvider, CapacityScheduler#setQueueAcls will set ACL one by one, so checkAccess's invoker can pull incomplete result.

        To avoid this, one possible solute might be: add a reintialize_lock to CS, it will be locked when doing reinitialize, and checkAccess will try to acquire it.

        Show
        leftnoteasy Wangda Tan added a comment - Jason Lowe , removing lock of checkAccess may cause consumer get partial queue's ACL when reinitialize happens. See YarnAuthorizationProvider , CapacityScheduler#setQueueAcls will set ACL one by one, so checkAccess's invoker can pull incomplete result. To avoid this, one possible solute might be: add a reintialize_lock to CS, it will be locked when doing reinitialize, and checkAccess will try to acquire it.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12725412/YARN-3487.002.patch
        against trunk revision fddd552.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

        org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup
        org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
        org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher
        org.apache.hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
        org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterService
        org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens
        org.apache.hadoop.yarn.server.resourcemanager.TestClientRMTokens

        Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7338//testReport/
        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7338//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12725412/YARN-3487.002.patch against trunk revision fddd552. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher org.apache.hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterService org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens org.apache.hadoop.yarn.server.resourcemanager.TestClientRMTokens Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7338//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7338//console This message is automatically generated.
        Hide
        sunilg Sunil G added a comment -

        Hi Wangda Tan and Jason Lowe
        During reloading CS config, if a queue s removed, eventhough 'queues' is a concurrent one, operation for delete may be in progress. And here we may try for checkAccess and pass it with success.

        Show
        sunilg Sunil G added a comment - Hi Wangda Tan and Jason Lowe During reloading CS config, if a queue s removed, eventhough 'queues' is a concurrent one, operation for delete may be in progress. And here we may try for checkAccess and pass it with success.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Sunil G, Sorry I didn't get what you meant, could you elaborate it?

        Show
        leftnoteasy Wangda Tan added a comment - Sunil G , Sorry I didn't get what you meant, could you elaborate it?
        Hide
        sunilg Sunil G added a comment -

        Hi Wangda Tan
        I am sorry for providing lesser content earlier. After seeing your comment again, i could see that my comment also was going on same line.

        Runtime updates can add or change some CLIs for a Queue. So if synchronized keyword s removed, checkAccess is open and some checks may pass/fail as per the partial information available for CLI of Queue.
        So we may run into partial errors which is a race case condition.

        Show
        sunilg Sunil G added a comment - Hi Wangda Tan I am sorry for providing lesser content earlier. After seeing your comment again, i could see that my comment also was going on same line. Runtime updates can add or change some CLIs for a Queue. So if synchronized keyword s removed, checkAccess is open and some checks may pass/fail as per the partial information available for CLI of Queue. So we may run into partial errors which is a race case condition.
        Hide
        jlowe Jason Lowe added a comment -

        Thanks for the feedback, Wangda and Sunil. In the interest of keeping this JIRA simple to expedite the getQueueInfo and getQueue fix this version of the patch restores the lock on checkAccess. IIRC there's already another JIRA proposing to add read/write locks to the CapacityScheduler to handle rare events like queue config refresh.

        Show
        jlowe Jason Lowe added a comment - Thanks for the feedback, Wangda and Sunil. In the interest of keeping this JIRA simple to expedite the getQueueInfo and getQueue fix this version of the patch restores the lock on checkAccess. IIRC there's already another JIRA proposing to add read/write locks to the CapacityScheduler to handle rare events like queue config refresh.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Thanks for feedback from Sunil G, Jason Lowe.

        Make this as a sub JIRA of YARN-3091, and w/r lock for CS is tracked by YARN-3139.

        The latest patch LGTM, will commit when Jenkins get back.

        Show
        leftnoteasy Wangda Tan added a comment - Thanks for feedback from Sunil G , Jason Lowe . Make this as a sub JIRA of YARN-3091 , and w/r lock for CS is tracked by YARN-3139 . The latest patch LGTM, will commit when Jenkins get back.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12726214/YARN-3487.003.patch
        against trunk revision c6b5203.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 4 new or modified test files.

        -1 javac. The applied patch generated 1207 javac compiler warnings (more than the trunk's current 1181 warnings).

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

        org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
        org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler
        org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
        org.apache.hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher
        org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
        org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched
        org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
        org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
        org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStoreZKClientConnections
        org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
        org.apache.hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore
        org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStorePerf
        org.apache.hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils
        org.apache.hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens
        org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens
        org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart
        org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
        org.apache.hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore
        org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization

        The following test timeouts occurred in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

        org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRTestTests
        org.apache.hadoop.yarn.server.resourcemanager.TestRMRestaTesTests
        org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorizatTests
        org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels

        Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7380//testReport/
        Javac warnings: https://builds.apache.org/job/PreCommit-YARN-Build/7380//artifact/patchprocess/diffJavacWarnings.txt
        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7380//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726214/YARN-3487.003.patch against trunk revision c6b5203. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. -1 javac . The applied patch generated 1207 javac compiler warnings (more than the trunk's current 1181 warnings). +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps org.apache.hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStoreZKClientConnections org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore org.apache.hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStorePerf org.apache.hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils org.apache.hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA org.apache.hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization The following test timeouts occurred in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRTestTests org.apache.hadoop.yarn.server.resourcemanager.TestRMRestaTesTests org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorizatTests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7380//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-YARN-Build/7380//artifact/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7380//console This message is automatically generated.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Re-triggerred Jenkins

        Show
        leftnoteasy Wangda Tan added a comment - Re-triggerred Jenkins
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12726214/YARN-3487.003.patch
        against trunk revision d573f09.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

        org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart

        Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7387//testReport/
        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7387//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726214/YARN-3487.003.patch against trunk revision d573f09. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7387//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7387//console This message is automatically generated.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Committed to branch-2, branch-2.7, trunk. Thanks Jason Lowe and review from Sunil G!

        Show
        leftnoteasy Wangda Tan added a comment - Committed to branch-2, branch-2.7, trunk. Thanks Jason Lowe and review from Sunil G !
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #7610 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7610/)
        YARN-3487. CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) (wangda: rev f47a5763acd55cb0b3f16152c7f8df06ec0e09a9)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7610 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7610/ ) YARN-3487 . CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) (wangda: rev f47a5763acd55cb0b3f16152c7f8df06ec0e09a9) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #167 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/167/)
        YARN-3487. CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) (wangda: rev f47a5763acd55cb0b3f16152c7f8df06ec0e09a9)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #167 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/167/ ) YARN-3487 . CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) (wangda: rev f47a5763acd55cb0b3f16152c7f8df06ec0e09a9) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2099 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2099/)
        YARN-3487. CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) (wangda: rev f47a5763acd55cb0b3f16152c7f8df06ec0e09a9)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2099 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2099/ ) YARN-3487 . CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) (wangda: rev f47a5763acd55cb0b3f16152c7f8df06ec0e09a9) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #158 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/158/)
        YARN-3487. CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) (wangda: rev f47a5763acd55cb0b3f16152c7f8df06ec0e09a9)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #158 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/158/ ) YARN-3487 . CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) (wangda: rev f47a5763acd55cb0b3f16152c7f8df06ec0e09a9) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #901 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/901/)
        YARN-3487. CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) (wangda: rev f47a5763acd55cb0b3f16152c7f8df06ec0e09a9)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #901 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/901/ ) YARN-3487 . CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) (wangda: rev f47a5763acd55cb0b3f16152c7f8df06ec0e09a9) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #168 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/168/)
        YARN-3487. CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) (wangda: rev f47a5763acd55cb0b3f16152c7f8df06ec0e09a9)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #168 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/168/ ) YARN-3487 . CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) (wangda: rev f47a5763acd55cb0b3f16152c7f8df06ec0e09a9) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2117 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2117/)
        YARN-3487. CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) (wangda: rev f47a5763acd55cb0b3f16152c7f8df06ec0e09a9)

        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2117 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2117/ ) YARN-3487 . CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) (wangda: rev f47a5763acd55cb0b3f16152c7f8df06ec0e09a9) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        Pulled this into 2.6.1. Ran compilation before the push. Patch applied cleanly.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - Pulled this into 2.6.1. Ran compilation before the push. Patch applied cleanly.

          People

          • Assignee:
            jlowe Jason Lowe
            Reporter:
            jlowe Jason Lowe
          • Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development