Details

    • Target Version/s:

      Description

      Enhance locks in AbstractCSQueue/LeafQueue/ParentQueue, as mentioned in YARN-3091, a possible solution is using read/write lock. Other fine-graind locks for specific purposes / bugs should be addressed in separated tickets.

      1. YARN-3140.4.patch
        172 kB
        Wangda Tan
      2. YARN-3140.3.patch
        172 kB
        Wangda Tan
      3. YARN-3140.2.patch
        172 kB
        Wangda Tan
      4. YARN-3140.1.patch
        172 kB
        Wangda Tan

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10464 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10464/)
        YARN-3140. Improve locks in AbstractCSQueue/LeafQueue/ParentQueue. (jianhe: rev 2b66d9ec5bdaec7e6b278926fbb6f222c4e3afaa)

        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestContainerResizing.java
        • (edit) hadoop-yarn-project/hadoop-yarn/dev-support/findbugs-exclude.xml
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/PlanQueue.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ReservationQueue.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10464 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10464/ ) YARN-3140 . Improve locks in AbstractCSQueue/LeafQueue/ParentQueue. (jianhe: rev 2b66d9ec5bdaec7e6b278926fbb6f222c4e3afaa) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestContainerResizing.java (edit) hadoop-yarn-project/hadoop-yarn/dev-support/findbugs-exclude.xml (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/PlanQueue.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ReservationQueue.java
        Hide
        jianhe Jian He added a comment -

        Committed to trunk, branch-2, thanks Wangda !

        Show
        jianhe Jian He added a comment - Committed to trunk, branch-2, thanks Wangda !
        Hide
        leftnoteasy Wangda Tan added a comment -

        Unit test failures are not related. Javadocs warnings are also not related: I noticed a couple of tests failed because of Javadocs warning, will investigate why they happened.

        Show
        leftnoteasy Wangda Tan added a comment - Unit test failures are not related. Javadocs warnings are also not related: I noticed a couple of tests failed because of Javadocs warning, will investigate why they happened.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 12m 56s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 9s Maven dependency ordering for branch
        +1 mvninstall 7m 6s trunk passed
        +1 compile 2m 28s trunk passed
        +1 checkstyle 0m 41s trunk passed
        +1 mvnsite 3m 47s trunk passed
        +1 mvneclipse 0m 49s trunk passed
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn
        +1 findbugs 0m 58s trunk passed
        -1 javadoc 1m 22s hadoop-yarn in trunk failed.
        -1 javadoc 0m 23s hadoop-yarn-server-resourcemanager in trunk failed.
        0 mvndep 0m 11s Maven dependency ordering for patch
        +1 mvninstall 3m 23s the patch passed
        +1 compile 2m 46s the patch passed
        +1 javac 2m 46s the patch passed
        -1 checkstyle 0m 41s hadoop-yarn-project/hadoop-yarn: The patch generated 39 new + 91 unchanged - 57 fixed = 130 total (was 148)
        +1 mvnsite 4m 13s the patch passed
        +1 mvneclipse 0m 49s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn
        +1 findbugs 1m 15s the patch passed
        -1 javadoc 1m 6s hadoop-yarn in the patch failed.
        -1 javadoc 0m 19s hadoop-yarn-server-resourcemanager in the patch failed.
        -1 unit 20m 45s hadoop-yarn in the patch failed.
        -1 unit 34m 46s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        101m 57s



        Reason Tests
        Failed junit tests hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices
          hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
          hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829237/YARN-3140.4.patch
        JIRA Issue YARN-3140
        Optional Tests asflicense findbugs xml compile javac javadoc mvninstall mvnsite unit checkstyle
        uname Linux ef9c6e53d79e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / b8a30f2
        Default Java 1.8.0_101
        findbugs v3.0.0
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13149/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13149/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13149/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13149/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13149/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13149/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13149/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13149/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn.txt https://builds.apache.org/job/PreCommit-YARN-Build/13149/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13149/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13149/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 12m 56s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 9s Maven dependency ordering for branch +1 mvninstall 7m 6s trunk passed +1 compile 2m 28s trunk passed +1 checkstyle 0m 41s trunk passed +1 mvnsite 3m 47s trunk passed +1 mvneclipse 0m 49s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn +1 findbugs 0m 58s trunk passed -1 javadoc 1m 22s hadoop-yarn in trunk failed. -1 javadoc 0m 23s hadoop-yarn-server-resourcemanager in trunk failed. 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 3m 23s the patch passed +1 compile 2m 46s the patch passed +1 javac 2m 46s the patch passed -1 checkstyle 0m 41s hadoop-yarn-project/hadoop-yarn: The patch generated 39 new + 91 unchanged - 57 fixed = 130 total (was 148) +1 mvnsite 4m 13s the patch passed +1 mvneclipse 0m 49s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn +1 findbugs 1m 15s the patch passed -1 javadoc 1m 6s hadoop-yarn in the patch failed. -1 javadoc 0m 19s hadoop-yarn-server-resourcemanager in the patch failed. -1 unit 20m 45s hadoop-yarn in the patch failed. -1 unit 34m 46s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 101m 57s Reason Tests Failed junit tests hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices   hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829237/YARN-3140.4.patch JIRA Issue YARN-3140 Optional Tests asflicense findbugs xml compile javac javadoc mvninstall mvnsite unit checkstyle uname Linux ef9c6e53d79e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b8a30f2 Default Java 1.8.0_101 findbugs v3.0.0 javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13149/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13149/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13149/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13149/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/13149/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13149/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13149/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13149/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn.txt https://builds.apache.org/job/PreCommit-YARN-Build/13149/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13149/testReport/ modules C: hadoop-yarn-project/hadoop-yarn hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/13149/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        leftnoteasy Wangda Tan added a comment - - edited

        Rebased and attached ver.4 patch. Jian He

        Show
        leftnoteasy Wangda Tan added a comment - - edited Rebased and attached ver.4 patch. Jian He
        Hide
        jianhe Jian He added a comment -

        The conflict is due to log statement reformatted into a bad format, we should probably preserve that.

        Show
        jianhe Jian He added a comment - The conflict is due to log statement reformatted into a bad format, we should probably preserve that.
        Hide
        jianhe Jian He added a comment -

        Wangda Tan, the patch doesn't apply any more, could you rebase?

        Show
        jianhe Jian He added a comment - Wangda Tan , the patch doesn't apply any more, could you rebase?
        Hide
        leftnoteasy Wangda Tan added a comment -

        Thanks Jian He, all failed tests are not related to this patch.

        Show
        leftnoteasy Wangda Tan added a comment - Thanks Jian He , all failed tests are not related to this patch.
        Hide
        jianhe Jian He added a comment -

        lgtm, committing in a day if no comments from others.

        Show
        jianhe Jian He added a comment - lgtm, committing in a day if no comments from others.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 10s Maven dependency ordering for branch
        +1 mvninstall 7m 8s trunk passed
        +1 compile 2m 28s trunk passed
        +1 checkstyle 0m 40s trunk passed
        +1 mvnsite 3m 46s trunk passed
        +1 mvneclipse 0m 49s trunk passed
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn
        +1 findbugs 0m 59s trunk passed
        +1 javadoc 1m 47s trunk passed
        0 mvndep 0m 10s Maven dependency ordering for patch
        +1 mvninstall 2m 52s the patch passed
        +1 compile 2m 24s the patch passed
        +1 javac 2m 24s the patch passed
        -1 checkstyle 0m 41s hadoop-yarn-project/hadoop-yarn: The patch generated 40 new + 91 unchanged - 57 fixed = 131 total (was 148)
        +1 mvnsite 3m 43s the patch passed
        +1 mvneclipse 0m 44s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn
        +1 findbugs 1m 5s the patch passed
        +1 javadoc 1m 46s the patch passed
        -1 unit 20m 59s hadoop-yarn in the patch failed.
        +1 unit 37m 59s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        91m 30s



        Reason Tests
        Failed junit tests hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager
          hadoop.yarn.server.nodemanager.TestDefaultContainerExecutor
          hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12827903/YARN-3140.2.patch
        JIRA Issue YARN-3140
        Optional Tests asflicense findbugs xml compile javac javadoc mvninstall mvnsite unit checkstyle
        uname Linux dc4bc0736bd9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / fcbac00
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13116/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13116/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13116/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13116/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13116/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 7m 8s trunk passed +1 compile 2m 28s trunk passed +1 checkstyle 0m 40s trunk passed +1 mvnsite 3m 46s trunk passed +1 mvneclipse 0m 49s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn +1 findbugs 0m 59s trunk passed +1 javadoc 1m 47s trunk passed 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 2m 52s the patch passed +1 compile 2m 24s the patch passed +1 javac 2m 24s the patch passed -1 checkstyle 0m 41s hadoop-yarn-project/hadoop-yarn: The patch generated 40 new + 91 unchanged - 57 fixed = 131 total (was 148) +1 mvnsite 3m 43s the patch passed +1 mvneclipse 0m 44s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn +1 findbugs 1m 5s the patch passed +1 javadoc 1m 46s the patch passed -1 unit 20m 59s hadoop-yarn in the patch failed. +1 unit 37m 59s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 91m 30s Reason Tests Failed junit tests hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager   hadoop.yarn.server.nodemanager.TestDefaultContainerExecutor   hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12827903/YARN-3140.2.patch JIRA Issue YARN-3140 Optional Tests asflicense findbugs xml compile javac javadoc mvninstall mvnsite unit checkstyle uname Linux dc4bc0736bd9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / fcbac00 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13116/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13116/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13116/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13116/testReport/ modules C: hadoop-yarn-project/hadoop-yarn hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/13116/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Thanks Jian He for review,

        Addressed all comments, except:

        private synchronized CSAssignment assignContainersToChildQueues

        It is already protected by writelocks inside assignContainers, so there's no need to keep the writelock inside assignContainersToChildQueues

        Any other comments?

        (Uploaded ver.3 patch)

        Show
        leftnoteasy Wangda Tan added a comment - Thanks Jian He for review, Addressed all comments, except: private synchronized CSAssignment assignContainersToChildQueues It is already protected by writelocks inside assignContainers, so there's no need to keep the writelock inside assignContainersToChildQueues Any other comments? (Uploaded ver.3 patch)
        Hide
        jianhe Jian He added a comment -
        • Is this method not used ? If so, labelManager no need to be volatile, and remove this method
            @VisibleForTesting
            public void setNodeLabelManager(RMNodeLabelsManager mgr) {
              this.labelManager = mgr;
            }
            
        • pendingOrderingPolicy: no need to be volatile
        • This synchronized keyword is removed, but no write lock is added
            private synchronized CSAssignment assignContainersToChildQueues(
            
        Show
        jianhe Jian He added a comment - Is this method not used ? If so, labelManager no need to be volatile, and remove this method @VisibleForTesting public void setNodeLabelManager(RMNodeLabelsManager mgr) { this .labelManager = mgr; } pendingOrderingPolicy: no need to be volatile This synchronized keyword is removed, but no write lock is added private synchronized CSAssignment assignContainersToChildQueues(
        Hide
        leftnoteasy Wangda Tan added a comment - - edited

        Jian He/Jason Lowe could you take a look at this patch?

        Show
        leftnoteasy Wangda Tan added a comment - - edited Jian He / Jason Lowe could you take a look at this patch?
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 10s Maven dependency ordering for branch
        +1 mvninstall 7m 8s trunk passed
        +1 compile 2m 47s trunk passed
        +1 checkstyle 0m 41s trunk passed
        +1 mvnsite 3m 46s trunk passed
        +1 mvneclipse 0m 50s trunk passed
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn
        +1 findbugs 1m 0s trunk passed
        +1 javadoc 1m 45s trunk passed
        0 mvndep 0m 10s Maven dependency ordering for patch
        +1 mvninstall 2m 49s the patch passed
        +1 compile 2m 19s the patch passed
        +1 javac 2m 19s the patch passed
        -1 checkstyle 0m 37s hadoop-yarn-project/hadoop-yarn: The patch generated 40 new + 91 unchanged - 57 fixed = 131 total (was 148)
        +1 mvnsite 3m 30s the patch passed
        +1 mvneclipse 0m 43s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn
        +1 findbugs 1m 3s the patch passed
        +1 javadoc 1m 41s the patch passed
        -1 unit 20m 1s hadoop-yarn in the patch failed.
        +1 unit 37m 25s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        89m 49s



        Reason Tests
        Failed junit tests hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12827903/YARN-3140.2.patch
        JIRA Issue YARN-3140
        Optional Tests asflicense findbugs xml compile javac javadoc mvninstall mvnsite unit checkstyle
        uname Linux 681b1c40424b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / bee9f57
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13073/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13073/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13073/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13073/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13073/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 7m 8s trunk passed +1 compile 2m 47s trunk passed +1 checkstyle 0m 41s trunk passed +1 mvnsite 3m 46s trunk passed +1 mvneclipse 0m 50s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn +1 findbugs 1m 0s trunk passed +1 javadoc 1m 45s trunk passed 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 2m 49s the patch passed +1 compile 2m 19s the patch passed +1 javac 2m 19s the patch passed -1 checkstyle 0m 37s hadoop-yarn-project/hadoop-yarn: The patch generated 40 new + 91 unchanged - 57 fixed = 131 total (was 148) +1 mvnsite 3m 30s the patch passed +1 mvneclipse 0m 43s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn +1 findbugs 1m 3s the patch passed +1 javadoc 1m 41s the patch passed -1 unit 20m 1s hadoop-yarn in the patch failed. +1 unit 37m 25s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 89m 49s Reason Tests Failed junit tests hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12827903/YARN-3140.2.patch JIRA Issue YARN-3140 Optional Tests asflicense findbugs xml compile javac javadoc mvninstall mvnsite unit checkstyle uname Linux 681b1c40424b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / bee9f57 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13073/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13073/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13073/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13073/testReport/ modules C: hadoop-yarn-project/hadoop-yarn hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/13073/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Attached ver.2 patch fixed unit tests and findbugs

        Show
        leftnoteasy Wangda Tan added a comment - Attached ver.2 patch fixed unit tests and findbugs
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 25s trunk passed
        +1 compile 0m 41s trunk passed
        +1 checkstyle 0m 22s trunk passed
        +1 mvnsite 0m 37s trunk passed
        +1 mvneclipse 0m 18s trunk passed
        +1 findbugs 0m 57s trunk passed
        +1 javadoc 0m 21s trunk passed
        +1 mvninstall 0m 34s the patch passed
        +1 compile 0m 30s the patch passed
        +1 javac 0m 30s the patch passed
        -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 40 new + 82 unchanged - 57 fixed = 122 total (was 139)
        +1 mvnsite 0m 34s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        -1 findbugs 1m 3s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 6 new + 0 unchanged - 0 fixed = 6 total (was 0)
        +1 javadoc 0m 18s the patch passed
        -1 unit 24m 17s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        39m 48s



        Reason Tests
        FindBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Unread field:LeafQueue.java:[line 1824]
          Increment of volatile field org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.activeApplications in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.activateApplication() At LeafQueue.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.activateApplication() At LeafQueue.java:[line 1905]
          Increment of volatile field org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.pendingApplications in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.activateApplication() At LeafQueue.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.activateApplication() At LeafQueue.java:[line 1904]
          Increment of volatile field org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.activeApplications in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.finishApplication(boolean) At LeafQueue.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.finishApplication(boolean) At LeafQueue.java:[line 1915]
          Increment of volatile field org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.pendingApplications in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.finishApplication(boolean) At LeafQueue.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.finishApplication(boolean) At LeafQueue.java:[line 1917]
          Increment of volatile field org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.pendingApplications in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.submitApplication() At LeafQueue.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.submitApplication() At LeafQueue.java:[line 1895]
        Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestParentQueue
          hadoop.yarn.server.resourcemanager.reservation.TestCapacitySchedulerPlanFollower
          hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerHealth
          hadoop.yarn.server.resourcemanager.TestResourceManager
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestChildQueueOrder
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue
          hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationLimits
        Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup
          org.apache.hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter
          org.apache.hadoop.yarn.server.resourcemanager.TestSignalContainer
          org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl
          org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler
          org.apache.hadoop.yarn.server.resourcemanager.TestRM
          org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched
          org.apache.hadoop.yarn.server.resourcemanager.TestRMHA
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
          org.apache.hadoop.yarn.server.resourcemanager.TestNodeBlacklistingOnAMFailures
          org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCResponseId
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.TestSchedulingWithAllocationRequestId
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationLimitsByPartition
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerSurgicalPreemption
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
          org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
          org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterService
          org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerLazyPreemption
          org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel
          org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher
          org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService
          org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesSchedulerActivities
          org.apache.hadoop.yarn.server.resourcemanager.TestDecommissioningNodesWatcher
          org.apache.hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
          org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
          org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
          org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCNodeUpdates
          org.apache.hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer
          org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.policy.TestFairOrderingPolicy
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
          org.apache.hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens
          org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate
          org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart
          org.apache.hadoop.yarn.server.resourcemanager.rmapp.TestNodesListManager
          org.apache.hadoop.yarn.server.resourcemanager.security.TestAMRMTokens
          org.apache.hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12827863/YARN-3140.1.patch
        JIRA Issue YARN-3140
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux ae6e556c84bd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / bee9f57
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13067/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        findbugs https://builds.apache.org/job/PreCommit-YARN-Build/13067/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13067/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13067/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13067/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13067/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 25s trunk passed +1 compile 0m 41s trunk passed +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 37s trunk passed +1 mvneclipse 0m 18s trunk passed +1 findbugs 0m 57s trunk passed +1 javadoc 0m 21s trunk passed +1 mvninstall 0m 34s the patch passed +1 compile 0m 30s the patch passed +1 javac 0m 30s the patch passed -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 40 new + 82 unchanged - 57 fixed = 122 total (was 139) +1 mvnsite 0m 34s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 1m 3s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 6 new + 0 unchanged - 0 fixed = 6 total (was 0) +1 javadoc 0m 18s the patch passed -1 unit 24m 17s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 39m 48s Reason Tests FindBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager   Unread field:LeafQueue.java: [line 1824]   Increment of volatile field org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.activeApplications in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.activateApplication() At LeafQueue.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.activateApplication() At LeafQueue.java: [line 1905]   Increment of volatile field org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.pendingApplications in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.activateApplication() At LeafQueue.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.activateApplication() At LeafQueue.java: [line 1904]   Increment of volatile field org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.activeApplications in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.finishApplication(boolean) At LeafQueue.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.finishApplication(boolean) At LeafQueue.java: [line 1915]   Increment of volatile field org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.pendingApplications in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.finishApplication(boolean) At LeafQueue.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.finishApplication(boolean) At LeafQueue.java: [line 1917]   Increment of volatile field org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.pendingApplications in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.submitApplication() At LeafQueue.java:in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User.submitApplication() At LeafQueue.java: [line 1895] Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestParentQueue   hadoop.yarn.server.resourcemanager.reservation.TestCapacitySchedulerPlanFollower   hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerHealth   hadoop.yarn.server.resourcemanager.TestResourceManager   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestChildQueueOrder   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationLimits Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup   org.apache.hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter   org.apache.hadoop.yarn.server.resourcemanager.TestSignalContainer   org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl   org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority   org.apache.hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler   org.apache.hadoop.yarn.server.resourcemanager.TestRM   org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched   org.apache.hadoop.yarn.server.resourcemanager.TestRMHA   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation   org.apache.hadoop.yarn.server.resourcemanager.TestNodeBlacklistingOnAMFailures   org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCResponseId   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior   org.apache.hadoop.yarn.server.resourcemanager.scheduler.TestSchedulingWithAllocationRequestId   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationLimitsByPartition   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerSurgicalPreemption   org.apache.hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation   org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterService   org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerLazyPreemption   org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel   org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher   org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService   org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesSchedulerActivities   org.apache.hadoop.yarn.server.resourcemanager.TestDecommissioningNodesWatcher   org.apache.hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA   org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization   org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCNodeUpdates   org.apache.hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer   org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   org.apache.hadoop.yarn.server.resourcemanager.scheduler.policy.TestFairOrderingPolicy   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs   org.apache.hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens   org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate   org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart   org.apache.hadoop.yarn.server.resourcemanager.rmapp.TestNodesListManager   org.apache.hadoop.yarn.server.resourcemanager.security.TestAMRMTokens   org.apache.hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12827863/YARN-3140.1.patch JIRA Issue YARN-3140 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ae6e556c84bd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / bee9f57 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13067/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/13067/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html unit https://builds.apache.org/job/PreCommit-YARN-Build/13067/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13067/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13067/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/13067/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Attached ver.1 patch for review.

        Show
        leftnoteasy Wangda Tan added a comment - Attached ver.1 patch for review.

          People

          • Assignee:
            leftnoteasy Wangda Tan
            Reporter:
            leftnoteasy Wangda Tan
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development