Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4983

JVM and UGI metrics disappear after RM is once transitioned to standby mode

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      When get transitioned to standby, the RM will shutdown the existing metric system and relaunch a new one. This will cause the jvm metrics and ugi metrics to miss in the new metric system.

      1. YARN-4983-trunk.003.patch
        10 kB
        Li Lu
      2. YARN-4983-trunk.002.patch
        10 kB
        Li Lu
      3. YARN-4983-trunk.001.patch
        6 kB
        Li Lu
      4. YARN-4983-trunk.000.patch
        6 kB
        Li Lu

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9678 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9678/)
          YARN-4983. JVM and UGI metrics disappear after RM transitioned to (jianhe: rev 4beff013546dbc29c004a1bad1b019dc9d2b751f)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/source/JvmMetrics.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMHAMetrics.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9678 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9678/ ) YARN-4983 . JVM and UGI metrics disappear after RM transitioned to (jianhe: rev 4beff013546dbc29c004a1bad1b019dc9d2b751f) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/source/JvmMetrics.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMHAMetrics.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
          Hide
          jianhe Jian He added a comment -

          Committed to trunk, branch-2, thanks Li !

          Show
          jianhe Jian He added a comment - Committed to trunk, branch-2, thanks Li !
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 8s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 14s Maven dependency ordering for branch
          +1 mvninstall 6m 57s trunk passed
          +1 compile 6m 36s trunk passed with JDK v1.8.0_92
          +1 compile 7m 4s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 8s trunk passed
          +1 mvnsite 1m 37s trunk passed
          +1 mvneclipse 0m 31s trunk passed
          +1 findbugs 2m 45s trunk passed
          +1 javadoc 1m 19s trunk passed with JDK v1.8.0_92
          +1 javadoc 1m 33s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 1m 17s the patch passed
          +1 compile 6m 34s the patch passed with JDK v1.8.0_92
          +1 javac 6m 34s the patch passed
          +1 compile 7m 7s the patch passed with JDK v1.7.0_95
          +1 javac 7m 7s the patch passed
          -1 checkstyle 1m 6s root: patch generated 1 new + 172 unchanged - 1 fixed = 173 total (was 173)
          +1 mvnsite 1m 33s the patch passed
          +1 mvneclipse 0m 27s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 3m 4s the patch passed
          +1 javadoc 1m 19s the patch passed with JDK v1.8.0_92
          +1 javadoc 1m 28s the patch passed with JDK v1.7.0_95
          +1 unit 7m 54s hadoop-common in the patch passed with JDK v1.8.0_92.
          -1 unit 77m 31s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_92.
          +1 unit 8m 28s hadoop-common in the patch passed with JDK v1.7.0_95.
          -1 unit 77m 31s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 21s Patch does not generate ASF License warnings.
          227m 9s



          Reason Tests
          JDK v1.8.0_92 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.8.0_92 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800885/YARN-4983-trunk.003.patch
          JIRA Issue YARN-4983
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 99e2fac815ef 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6be22dd
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_92 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11229/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11229/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_92.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11229/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11229/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_92.txt https://builds.apache.org/job/PreCommit-YARN-Build/11229/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11229/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: .
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11229/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 8s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 6m 57s trunk passed +1 compile 6m 36s trunk passed with JDK v1.8.0_92 +1 compile 7m 4s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 8s trunk passed +1 mvnsite 1m 37s trunk passed +1 mvneclipse 0m 31s trunk passed +1 findbugs 2m 45s trunk passed +1 javadoc 1m 19s trunk passed with JDK v1.8.0_92 +1 javadoc 1m 33s trunk passed with JDK v1.7.0_95 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 17s the patch passed +1 compile 6m 34s the patch passed with JDK v1.8.0_92 +1 javac 6m 34s the patch passed +1 compile 7m 7s the patch passed with JDK v1.7.0_95 +1 javac 7m 7s the patch passed -1 checkstyle 1m 6s root: patch generated 1 new + 172 unchanged - 1 fixed = 173 total (was 173) +1 mvnsite 1m 33s the patch passed +1 mvneclipse 0m 27s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 3m 4s the patch passed +1 javadoc 1m 19s the patch passed with JDK v1.8.0_92 +1 javadoc 1m 28s the patch passed with JDK v1.7.0_95 +1 unit 7m 54s hadoop-common in the patch passed with JDK v1.8.0_92. -1 unit 77m 31s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_92. +1 unit 8m 28s hadoop-common in the patch passed with JDK v1.7.0_95. -1 unit 77m 31s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 227m 9s Reason Tests JDK v1.8.0_92 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.8.0_92 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800885/YARN-4983-trunk.003.patch JIRA Issue YARN-4983 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 99e2fac815ef 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6be22dd Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_92 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11229/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11229/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_92.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11229/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11229/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_92.txt https://builds.apache.org/job/PreCommit-YARN-Build/11229/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11229/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/11229/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          gtCarrera9 Li Lu added a comment -

          New patch, removed the wrong exception tag in javadoc. Kicking Jenkins for another round on all tests.

          Show
          gtCarrera9 Li Lu added a comment - New patch, removed the wrong exception tag in javadoc. Kicking Jenkins for another round on all tests.
          Hide
          gtCarrera9 Li Lu added a comment -

          One pedantic little comment: lest we work against YARN-4977, can we please include a message on the throws tag in the javadoc headers?

          Sure. I'll update that part.

          Also, it looks like the test tests the transition from active to standby, but not back to active. Any value in adding that last step?

          Not necessarily? The problem happens when we try to reinitialize the RM active service. This step happens when we call transitionToStandBy. The actual symptom also happens when the RM becomes standby from active.

          Show
          gtCarrera9 Li Lu added a comment - One pedantic little comment: lest we work against YARN-4977 , can we please include a message on the throws tag in the javadoc headers? Sure. I'll update that part. Also, it looks like the test tests the transition from active to standby, but not back to active. Any value in adding that last step? Not necessarily? The problem happens when we try to reinitialize the RM active service. This step happens when we call transitionToStandBy. The actual symptom also happens when the RM becomes standby from active.
          Hide
          templedf Daniel Templeton added a comment -

          One pedantic little comment: lest we work against YARN-4977, can we please include a message on the throws tag in the javadoc headers?

          Also, it looks like the test tests the transition from active to standby, but not back to active. Any value in adding that last step?

          Show
          templedf Daniel Templeton added a comment - One pedantic little comment: lest we work against YARN-4977 , can we please include a message on the throws tag in the javadoc headers? Also, it looks like the test tests the transition from active to standby, but not back to active. Any value in adding that last step?
          Hide
          gtCarrera9 Li Lu added a comment -

          I thought Daniel Templeton uploaded a patch to another JIRA to this one and then deleted it? Just try to confirm... Thanks.

          Show
          gtCarrera9 Li Lu added a comment - I thought Daniel Templeton uploaded a patch to another JIRA to this one and then deleted it? Just try to confirm... Thanks.
          Hide
          djp Junping Du added a comment -

          Hi Daniel Templeton, do you have more comments on the latest patch? If not, I will go ahead to commit this patch which is an important bug fix.

          Show
          djp Junping Du added a comment - Hi Daniel Templeton , do you have more comments on the latest patch? If not, I will go ahead to commit this patch which is an important bug fix.
          Hide
          djp Junping Du added a comment -

          Due to the amount of work to refactor RM HA, maybe we want to resolve that in a separate JIRA?

          Sounds good. Will file a separate JIRA to track this.

          Three UT test failures are not related, HADOOP-12687 and YARN-4947 are filed before to track them.

          +1. Patch LGTM. Will commit it shortly if no further comments from others.

          Show
          djp Junping Du added a comment - Due to the amount of work to refactor RM HA, maybe we want to resolve that in a separate JIRA? Sounds good. Will file a separate JIRA to track this. Three UT test failures are not related, HADOOP-12687 and YARN-4947 are filed before to track them. +1. Patch LGTM. Will commit it shortly if no further comments from others.
          Hide
          gtCarrera9 Li Lu added a comment -

          Could not reproduce the UT failures locally. At least two UT failures (TestClientRMTokens and TestAMAuthorization) also appears in other unrelated JIRAs, such as YARN-3150 (a documentation patch).

          Show
          gtCarrera9 Li Lu added a comment - Could not reproduce the UT failures locally. At least two UT failures (TestClientRMTokens and TestAMAuthorization) also appears in other unrelated JIRAs, such as YARN-3150 (a documentation patch).
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 6m 52s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 49s Maven dependency ordering for branch
          +1 mvninstall 7m 5s trunk passed
          +1 compile 6m 18s trunk passed with JDK v1.8.0_92
          +1 compile 6m 59s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 7s trunk passed
          +1 mvnsite 1m 30s trunk passed
          +1 mvneclipse 0m 29s trunk passed
          +1 findbugs 2m 43s trunk passed
          +1 javadoc 1m 13s trunk passed with JDK v1.8.0_92
          +1 javadoc 1m 30s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 1m 9s the patch passed
          +1 compile 5m 38s the patch passed with JDK v1.8.0_92
          +1 javac 5m 38s the patch passed
          +1 compile 6m 32s the patch passed with JDK v1.7.0_95
          +1 javac 6m 32s the patch passed
          -1 checkstyle 1m 5s root: patch generated 1 new + 172 unchanged - 1 fixed = 173 total (was 173)
          +1 mvnsite 1m 28s the patch passed
          +1 mvneclipse 0m 29s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 3m 4s the patch passed
          -1 javadoc 0m 21s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_92.
          +1 javadoc 1m 27s the patch passed with JDK v1.7.0_95
          +1 unit 6m 48s hadoop-common in the patch passed with JDK v1.8.0_92.
          -1 unit 76m 0s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_92.
          +1 unit 7m 3s hadoop-common in the patch passed with JDK v1.7.0_95.
          -1 unit 77m 10s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 23s Patch does not generate ASF License warnings.
          227m 42s



          Reason Tests
          JDK v1.8.0_92 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.8.0_92 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800345/YARN-4983-trunk.002.patch
          JIRA Issue YARN-4983
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux c9129d661fc3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 10f0f78
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_92 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11207/artifact/patchprocess/diff-checkstyle-root.txt
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/11207/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_92.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11207/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_92.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11207/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11207/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_92.txt https://builds.apache.org/job/PreCommit-YARN-Build/11207/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11207/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: .
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11207/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 6m 52s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 49s Maven dependency ordering for branch +1 mvninstall 7m 5s trunk passed +1 compile 6m 18s trunk passed with JDK v1.8.0_92 +1 compile 6m 59s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 7s trunk passed +1 mvnsite 1m 30s trunk passed +1 mvneclipse 0m 29s trunk passed +1 findbugs 2m 43s trunk passed +1 javadoc 1m 13s trunk passed with JDK v1.8.0_92 +1 javadoc 1m 30s trunk passed with JDK v1.7.0_95 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 9s the patch passed +1 compile 5m 38s the patch passed with JDK v1.8.0_92 +1 javac 5m 38s the patch passed +1 compile 6m 32s the patch passed with JDK v1.7.0_95 +1 javac 6m 32s the patch passed -1 checkstyle 1m 5s root: patch generated 1 new + 172 unchanged - 1 fixed = 173 total (was 173) +1 mvnsite 1m 28s the patch passed +1 mvneclipse 0m 29s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 3m 4s the patch passed -1 javadoc 0m 21s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_92. +1 javadoc 1m 27s the patch passed with JDK v1.7.0_95 +1 unit 6m 48s hadoop-common in the patch passed with JDK v1.8.0_92. -1 unit 76m 0s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_92. +1 unit 7m 3s hadoop-common in the patch passed with JDK v1.7.0_95. -1 unit 77m 10s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 227m 42s Reason Tests JDK v1.8.0_92 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.8.0_92 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800345/YARN-4983-trunk.002.patch JIRA Issue YARN-4983 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c9129d661fc3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 10f0f78 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_92 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11207/artifact/patchprocess/diff-checkstyle-root.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/11207/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_92.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11207/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_92.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11207/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11207/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_92.txt https://builds.apache.org/job/PreCommit-YARN-Build/11207/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11207/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/11207/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          gtCarrera9 Li Lu added a comment -

          Thanks for the review Junping Du! I totally agree that we need to refactor the code of RM HA so that the code is more maintainable. For the boolean flag here, initialize and fromActive are currently decoupled. createAndInitActiveServices has two call sites, one in RM#serviceInit, another in reinitialize. When reinitializing, we're using the flag initialize to distinguish if we need to reinitialize the active service. This method is called when we do a state transition to standby and active. Therefore, we only need to set fromActive to true when we're reinitializing the active services and is transitioning from active to standby. Due to the amount of work to refactor RM HA, maybe we want to resolve that in a separate JIRA? We can focus on a quick fix for the current symptom in this JIRA? Thanks!

          BTW, I kicked another Jenkins run to see if there are any other UT problems.

          Show
          gtCarrera9 Li Lu added a comment - Thanks for the review Junping Du ! I totally agree that we need to refactor the code of RM HA so that the code is more maintainable. For the boolean flag here, initialize and fromActive are currently decoupled. createAndInitActiveServices has two call sites, one in RM#serviceInit, another in reinitialize. When reinitializing, we're using the flag initialize to distinguish if we need to reinitialize the active service. This method is called when we do a state transition to standby and active. Therefore, we only need to set fromActive to true when we're reinitializing the active services and is transitioning from active to standby. Due to the amount of work to refactor RM HA, maybe we want to resolve that in a separate JIRA? We can focus on a quick fix for the current symptom in this JIRA? Thanks! BTW, I kicked another Jenkins run to see if there are any other UT problems.
          Hide
          djp Junping Du added a comment -

          Thanks Li Lu for reporting the issue and delivering the fix.
          The patch looks good in overall. There are some questions and comments below.
          I think our work here is trying to differentiate the metric initialization/re-initialization with RM first time start with RM transition from standby. Isn't it? If so, below code:

          @@ -1114,7 +1127,7 @@ void reinitialize(boolean initialize) {
               QueueMetrics.clearQueueMetrics();
               if (initialize) {
                 resetDispatcher();
          -      createAndInitActiveServices();
          +      createAndInitActiveServices(true);
               }
             }
          

          Shall we reuse boolean initialize instead of set true directly? I saw there are two caller path: one is from RM.serviceStart() and the other is from transition to standby from active, marked by boolean value of initialize. I think we should also differentiate the situation here?

          In addition, I think our RM start logic is a bit redundant when RM HA is enabled: we always do serviceInit() first, then serviceStart(). However, within serviceStart(), we always put RM to standby mode first and do reinitialize again. Any special reason that we cannot initialize once to be in standby mode?
          CC Jian He, Karthik Kambatla.

          Show
          djp Junping Du added a comment - Thanks Li Lu for reporting the issue and delivering the fix. The patch looks good in overall. There are some questions and comments below. I think our work here is trying to differentiate the metric initialization/re-initialization with RM first time start with RM transition from standby. Isn't it? If so, below code: @@ -1114,7 +1127,7 @@ void reinitialize(boolean initialize) { QueueMetrics.clearQueueMetrics(); if (initialize) { resetDispatcher(); - createAndInitActiveServices(); + createAndInitActiveServices(true); } } Shall we reuse boolean initialize instead of set true directly? I saw there are two caller path: one is from RM.serviceStart() and the other is from transition to standby from active, marked by boolean value of initialize. I think we should also differentiate the situation here? In addition, I think our RM start logic is a bit redundant when RM HA is enabled: we always do serviceInit() first, then serviceStart(). However, within serviceStart(), we always put RM to standby mode first and do reinitialize again. Any special reason that we cannot initialize once to be in standby mode? CC Jian He , Karthik Kambatla .
          Hide
          gtCarrera9 Li Lu added a comment -

          Added an unit test for the standby metrics. The UT failures in the previous run appears to be independent to the changes in this patch. Trying them one more time.

          Show
          gtCarrera9 Li Lu added a comment - Added an unit test for the standby metrics. The UT failures in the previous run appears to be independent to the changes in this patch. Trying them one more time.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 15s Maven dependency ordering for branch
          +1 mvninstall 7m 51s trunk passed
          +1 compile 9m 29s trunk passed with JDK v1.8.0_77
          +1 compile 8m 8s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 11s trunk passed
          +1 mvnsite 1m 45s trunk passed
          +1 mvneclipse 0m 31s trunk passed
          +1 findbugs 2m 53s trunk passed
          +1 javadoc 1m 39s trunk passed with JDK v1.8.0_77
          +1 javadoc 1m 43s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 1m 21s the patch passed
          +1 compile 9m 12s the patch passed with JDK v1.8.0_77
          +1 javac 9m 12s the patch passed
          +1 compile 8m 6s the patch passed with JDK v1.7.0_95
          +1 javac 8m 6s the patch passed
          -1 checkstyle 1m 11s root: patch generated 1 new + 173 unchanged - 1 fixed = 174 total (was 174)
          +1 mvnsite 1m 39s the patch passed
          +1 mvneclipse 0m 30s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 3m 26s the patch passed
          -1 javadoc 0m 31s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77.
          +1 javadoc 1m 40s the patch passed with JDK v1.7.0_95
          +1 unit 10m 50s hadoop-common in the patch passed with JDK v1.8.0_77.
          -1 unit 83m 39s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77.
          -1 unit 9m 55s hadoop-common in the patch failed with JDK v1.7.0_95.
          -1 unit 86m 46s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 36s Patch does not generate ASF License warnings.
          257m 57s



          Reason Tests
          JDK v1.8.0_77 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
          JDK v1.7.0_95 Failed junit tests hadoop.security.ssl.TestReloadingX509TrustManager
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800162/YARN-4983-trunk.001.patch
          JIRA Issue YARN-4983
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 3b7cac4bb965 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 7cb3a3d
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11181/artifact/patchprocess/diff-checkstyle-root.txt
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/11181/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11181/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11181/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11181/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11181/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11181/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-YARN-Build/11181/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11181/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: .
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11181/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 7m 51s trunk passed +1 compile 9m 29s trunk passed with JDK v1.8.0_77 +1 compile 8m 8s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 11s trunk passed +1 mvnsite 1m 45s trunk passed +1 mvneclipse 0m 31s trunk passed +1 findbugs 2m 53s trunk passed +1 javadoc 1m 39s trunk passed with JDK v1.8.0_77 +1 javadoc 1m 43s trunk passed with JDK v1.7.0_95 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 1m 21s the patch passed +1 compile 9m 12s the patch passed with JDK v1.8.0_77 +1 javac 9m 12s the patch passed +1 compile 8m 6s the patch passed with JDK v1.7.0_95 +1 javac 8m 6s the patch passed -1 checkstyle 1m 11s root: patch generated 1 new + 173 unchanged - 1 fixed = 174 total (was 174) +1 mvnsite 1m 39s the patch passed +1 mvneclipse 0m 30s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 3m 26s the patch passed -1 javadoc 0m 31s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77. +1 javadoc 1m 40s the patch passed with JDK v1.7.0_95 +1 unit 10m 50s hadoop-common in the patch passed with JDK v1.8.0_77. -1 unit 83m 39s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77. -1 unit 9m 55s hadoop-common in the patch failed with JDK v1.7.0_95. -1 unit 86m 46s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 36s Patch does not generate ASF License warnings. 257m 57s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes JDK v1.7.0_95 Failed junit tests hadoop.security.ssl.TestReloadingX509TrustManager   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800162/YARN-4983-trunk.001.patch JIRA Issue YARN-4983 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 3b7cac4bb965 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 7cb3a3d Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11181/artifact/patchprocess/diff-checkstyle-root.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/11181/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11181/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11181/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11181/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11181/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11181/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-YARN-Build/11181/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11181/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/11181/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          gtCarrera9 Li Lu added a comment -

          Similar UT failures happened in HADOOP-12563. The patch now get reverted. I'm launching another Jenkins run for this JIRA.

          Show
          gtCarrera9 Li Lu added a comment - Similar UT failures happened in HADOOP-12563 . The patch now get reverted. I'm launching another Jenkins run for this JIRA.
          Hide
          templedf Daniel Templeton added a comment -

          I meant there are unit tests that are tripping on the problems you're trying to fix.

          Show
          templedf Daniel Templeton added a comment - I meant there are unit tests that are tripping on the problems you're trying to fix.
          Hide
          gtCarrera9 Li Lu added a comment -

          Thanks Daniel Templeton!

          Some of the unit tests that trip on it are only creating disembodied schedulers.

          Did you mean the failures for the first time, or from the second patch? I'm trying to understand with the second groups of failures, with something complaining from the protobuf level...

          Show
          gtCarrera9 Li Lu added a comment - Thanks Daniel Templeton ! Some of the unit tests that trip on it are only creating disembodied schedulers. Did you mean the failures for the first time, or from the second patch? I'm trying to understand with the second groups of failures, with something complaining from the protobuf level...
          Hide
          templedf Daniel Templeton added a comment -

          This also comes up as an issue with some unit tests. I agree that the root issue is the erroneous assumption that the RM and scheduler won't be started a second time from within the same VM. Ideally we'd fix it below the level or the RM. Some of the unit tests that trip on it are only creating disembodied schedulers.

          Show
          templedf Daniel Templeton added a comment - This also comes up as an issue with some unit tests. I agree that the root issue is the erroneous assumption that the RM and scheduler won't be started a second time from within the same VM. Ideally we'd fix it below the level or the RM. Some of the unit tests that trip on it are only creating disembodied schedulers.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 15s Maven dependency ordering for branch
          +1 mvninstall 6m 34s trunk passed
          +1 compile 5m 38s trunk passed with JDK v1.8.0_77
          +1 compile 6m 37s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 6s trunk passed
          +1 mvnsite 1m 31s trunk passed
          +1 mvneclipse 0m 29s trunk passed
          +1 findbugs 2m 42s trunk passed
          +1 javadoc 1m 14s trunk passed with JDK v1.8.0_77
          +1 javadoc 1m 33s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 15s Maven dependency ordering for patch
          +1 mvninstall 1m 11s the patch passed
          +1 compile 5m 36s the patch passed with JDK v1.8.0_77
          +1 javac 5m 36s the patch passed
          +1 compile 6m 37s the patch passed with JDK v1.7.0_95
          +1 javac 6m 37s the patch passed
          -1 checkstyle 1m 7s root: patch generated 1 new + 173 unchanged - 1 fixed = 174 total (was 174)
          +1 mvnsite 1m 31s the patch passed
          +1 mvneclipse 0m 29s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 3m 8s the patch passed
          -1 javadoc 0m 22s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77.
          +1 javadoc 1m 32s the patch passed with JDK v1.7.0_95
          -1 unit 6m 45s hadoop-common in the patch failed with JDK v1.8.0_77.
          -1 unit 80m 3s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77.
          -1 unit 7m 13s hadoop-common in the patch failed with JDK v1.7.0_95.
          -1 unit 81m 37s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 33s Patch does not generate ASF License warnings.
          228m 10s



          Reason Tests
          JDK v1.8.0_77 Failed junit tests hadoop.net.TestDNS
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestRMRestart
            hadoop.yarn.server.resourcemanager.TestAppManager
            hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
            hadoop.yarn.server.resourcemanager.TestApplicationCleanup
            hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebappAuthentication
            hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication
            hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority
          JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
          JDK v1.7.0_95 Failed junit tests hadoop.security.ssl.TestReloadingX509TrustManager
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestRMRestart
            hadoop.yarn.server.resourcemanager.TestAppManager
            hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
            hadoop.yarn.server.resourcemanager.TestApplicationCleanup
            hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebappAuthentication
            hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication
            hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800162/YARN-4983-trunk.001.patch
          JIRA Issue YARN-4983
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 5638befd5fbb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 337bcde
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11176/artifact/patchprocess/diff-checkstyle-root.txt
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/11176/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11176/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11176/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11176/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11176/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11176/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11176/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11176/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-YARN-Build/11176/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11176/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: .
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11176/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 6m 34s trunk passed +1 compile 5m 38s trunk passed with JDK v1.8.0_77 +1 compile 6m 37s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 6s trunk passed +1 mvnsite 1m 31s trunk passed +1 mvneclipse 0m 29s trunk passed +1 findbugs 2m 42s trunk passed +1 javadoc 1m 14s trunk passed with JDK v1.8.0_77 +1 javadoc 1m 33s trunk passed with JDK v1.7.0_95 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 1m 11s the patch passed +1 compile 5m 36s the patch passed with JDK v1.8.0_77 +1 javac 5m 36s the patch passed +1 compile 6m 37s the patch passed with JDK v1.7.0_95 +1 javac 6m 37s the patch passed -1 checkstyle 1m 7s root: patch generated 1 new + 173 unchanged - 1 fixed = 174 total (was 174) +1 mvnsite 1m 31s the patch passed +1 mvneclipse 0m 29s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 3m 8s the patch passed -1 javadoc 0m 22s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77. +1 javadoc 1m 32s the patch passed with JDK v1.7.0_95 -1 unit 6m 45s hadoop-common in the patch failed with JDK v1.8.0_77. -1 unit 80m 3s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77. -1 unit 7m 13s hadoop-common in the patch failed with JDK v1.7.0_95. -1 unit 81m 37s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 33s Patch does not generate ASF License warnings. 228m 10s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.net.TestDNS   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestRMRestart   hadoop.yarn.server.resourcemanager.TestAppManager   hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   hadoop.yarn.server.resourcemanager.TestApplicationCleanup   hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   hadoop.yarn.server.resourcemanager.webapp.TestRMWebappAuthentication   hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication   hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes JDK v1.7.0_95 Failed junit tests hadoop.security.ssl.TestReloadingX509TrustManager   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestRMRestart   hadoop.yarn.server.resourcemanager.TestAppManager   hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   hadoop.yarn.server.resourcemanager.TestApplicationCleanup   hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   hadoop.yarn.server.resourcemanager.webapp.TestRMWebappAuthentication   hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication   hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800162/YARN-4983-trunk.001.patch JIRA Issue YARN-4983 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 5638befd5fbb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 337bcde Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11176/artifact/patchprocess/diff-checkstyle-root.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/11176/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11176/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11176/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11176/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11176/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11176/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11176/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11176/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-YARN-Build/11176/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11176/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/11176/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          gtCarrera9 Li Lu added a comment -

          The tons of UT failures are caused by a wrong jvm metrics API call. Calling initSingleton instead of create to avoid registering twice to the same metrics system.

          Show
          gtCarrera9 Li Lu added a comment - The tons of UT failures are caused by a wrong jvm metrics API call. Calling initSingleton instead of create to avoid registering twice to the same metrics system.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 15s Maven dependency ordering for branch
          +1 mvninstall 6m 37s trunk passed
          +1 compile 6m 1s trunk passed with JDK v1.8.0_77
          +1 compile 6m 49s trunk passed with JDK v1.7.0_95
          +1 checkstyle 1m 6s trunk passed
          +1 mvnsite 1m 32s trunk passed
          +1 mvneclipse 0m 27s trunk passed
          +1 findbugs 2m 40s trunk passed
          +1 javadoc 1m 15s trunk passed with JDK v1.8.0_77
          +1 javadoc 1m 32s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 1m 12s the patch passed
          +1 compile 6m 7s the patch passed with JDK v1.8.0_77
          +1 javac 6m 7s the patch passed
          +1 compile 6m 43s the patch passed with JDK v1.7.0_95
          +1 javac 6m 43s the patch passed
          -1 checkstyle 1m 4s root: patch generated 2 new + 172 unchanged - 1 fixed = 174 total (was 173)
          +1 mvnsite 1m 30s the patch passed
          +1 mvneclipse 0m 28s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 3m 5s the patch passed
          -1 javadoc 0m 22s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77.
          +1 javadoc 1m 32s the patch passed with JDK v1.7.0_95
          -1 unit 7m 43s hadoop-common in the patch failed with JDK v1.8.0_77.
          -1 unit 68m 27s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77.
          +1 unit 8m 34s hadoop-common in the patch passed with JDK v1.7.0_95.
          -1 unit 69m 22s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 21s Patch does not generate ASF License warnings.
          207m 21s



          Reason Tests
          JDK v1.8.0_77 Failed junit tests hadoop.security.ssl.TestReloadingX509TrustManager
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
            hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
            hadoop.yarn.server.resourcemanager.TestRMHA
            hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestApplicationMasterService
            hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler
            hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
            hadoop.yarn.server.resourcemanager.TestLeaderElectorService
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler
            hadoop.yarn.server.resourcemanager.TestApplicationCleanup
            hadoop.yarn.server.resourcemanager.TestRMHAForNodeLabels
            hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior
            hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher
            hadoop.yarn.server.resourcemanager.TestMoveApplication
            hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions
            hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppAttempt
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebappAuthentication
            hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication
            hadoop.yarn.server.resourcemanager.rmapp.TestNodesListManager
            hadoop.yarn.webapp.TestRMWithCSRFFilter
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
            hadoop.yarn.server.resourcemanager.TestRMRestart
            hadoop.yarn.server.resourcemanager.TestRMAdminService
            hadoop.yarn.server.resourcemanager.TestResourceManager
            hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
            hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerHealth
            hadoop.yarn.server.resourcemanager.TestAppManager
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPreemption
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestReservationSystemWithRMHA
            hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl
          JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
            hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
            hadoop.yarn.server.resourcemanager.TestRMHA
            hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestApplicationMasterService
            hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler
            hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
            hadoop.yarn.server.resourcemanager.TestLeaderElectorService
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler
            hadoop.yarn.server.resourcemanager.TestApplicationCleanup
            hadoop.yarn.server.resourcemanager.TestRMHAForNodeLabels
            hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior
            hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher
            hadoop.yarn.server.resourcemanager.TestMoveApplication
            hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions
            hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppAttempt
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebappAuthentication
            hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication
            hadoop.yarn.server.resourcemanager.rmapp.TestNodesListManager
            hadoop.yarn.webapp.TestRMWithCSRFFilter
            hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
            hadoop.yarn.server.resourcemanager.TestRMRestart
            hadoop.yarn.server.resourcemanager.TestRMAdminService
            hadoop.yarn.server.resourcemanager.TestResourceManager
            hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
            hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerHealth
            hadoop.yarn.server.resourcemanager.TestAppManager
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPreemption
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestReservationSystemWithRMHA
            hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800090/YARN-4983-trunk.000.patch
          JIRA Issue YARN-4983
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 6f97042aa3b4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 14ab7a8
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11169/artifact/patchprocess/diff-checkstyle-root.txt
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/11169/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11169/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11169/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11169/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11169/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11169/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11169/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11169/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: .
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11169/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 6m 37s trunk passed +1 compile 6m 1s trunk passed with JDK v1.8.0_77 +1 compile 6m 49s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 6s trunk passed +1 mvnsite 1m 32s trunk passed +1 mvneclipse 0m 27s trunk passed +1 findbugs 2m 40s trunk passed +1 javadoc 1m 15s trunk passed with JDK v1.8.0_77 +1 javadoc 1m 32s trunk passed with JDK v1.7.0_95 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 12s the patch passed +1 compile 6m 7s the patch passed with JDK v1.8.0_77 +1 javac 6m 7s the patch passed +1 compile 6m 43s the patch passed with JDK v1.7.0_95 +1 javac 6m 43s the patch passed -1 checkstyle 1m 4s root: patch generated 2 new + 172 unchanged - 1 fixed = 174 total (was 173) +1 mvnsite 1m 30s the patch passed +1 mvneclipse 0m 28s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 3m 5s the patch passed -1 javadoc 0m 22s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77. +1 javadoc 1m 32s the patch passed with JDK v1.7.0_95 -1 unit 7m 43s hadoop-common in the patch failed with JDK v1.8.0_77. -1 unit 68m 27s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_77. +1 unit 8m 34s hadoop-common in the patch passed with JDK v1.7.0_95. -1 unit 69m 22s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 207m 21s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.security.ssl.TestReloadingX509TrustManager   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer   hadoop.yarn.server.resourcemanager.TestRMHA   hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestApplicationMasterService   hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler   hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler   hadoop.yarn.server.resourcemanager.TestLeaderElectorService   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler   hadoop.yarn.server.resourcemanager.TestApplicationCleanup   hadoop.yarn.server.resourcemanager.TestRMHAForNodeLabels   hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior   hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher   hadoop.yarn.server.resourcemanager.TestMoveApplication   hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppAttempt   hadoop.yarn.server.resourcemanager.webapp.TestRMWebappAuthentication   hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication   hadoop.yarn.server.resourcemanager.rmapp.TestNodesListManager   hadoop.yarn.webapp.TestRMWithCSRFFilter   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices   hadoop.yarn.server.resourcemanager.TestRMRestart   hadoop.yarn.server.resourcemanager.TestRMAdminService   hadoop.yarn.server.resourcemanager.TestResourceManager   hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerHealth   hadoop.yarn.server.resourcemanager.TestAppManager   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPreemption   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestReservationSystemWithRMHA   hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl JDK v1.8.0_77 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes JDK v1.7.0_95 Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer   hadoop.yarn.server.resourcemanager.TestRMHA   hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestApplicationMasterService   hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler   hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler   hadoop.yarn.server.resourcemanager.TestLeaderElectorService   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler   hadoop.yarn.server.resourcemanager.TestApplicationCleanup   hadoop.yarn.server.resourcemanager.TestRMHAForNodeLabels   hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior   hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher   hadoop.yarn.server.resourcemanager.TestMoveApplication   hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppAttempt   hadoop.yarn.server.resourcemanager.webapp.TestRMWebappAuthentication   hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication   hadoop.yarn.server.resourcemanager.rmapp.TestNodesListManager   hadoop.yarn.webapp.TestRMWithCSRFFilter   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices   hadoop.yarn.server.resourcemanager.TestRMRestart   hadoop.yarn.server.resourcemanager.TestRMAdminService   hadoop.yarn.server.resourcemanager.TestResourceManager   hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerHealth   hadoop.yarn.server.resourcemanager.TestAppManager   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPreemption   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestReservationSystemWithRMHA   hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800090/YARN-4983-trunk.000.patch JIRA Issue YARN-4983 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 6f97042aa3b4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 14ab7a8 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/11169/artifact/patchprocess/diff-checkstyle-root.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/11169/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11169/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11169/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11169/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11169/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11169/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-YARN-Build/11169/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11169/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/11169/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          gtCarrera9 Li Lu added a comment -

          This is a patch to quickly fix the problem. I'm reattaching the JvmMetrics after the rm got transitioned to standby state. UgiMetrics is a different story since it fails when get reattached. As a quick fix I recreate it.

          Right now this patch is just demo for a quick fix. I think the root contradiction here is our metric system does not quite consider the case when a metric system got restarted (and metrics got reattached), while the RM HA code relaunches the metric system. If there are more elegant ways to fix this please do let me know. Thanks!

          Show
          gtCarrera9 Li Lu added a comment - This is a patch to quickly fix the problem. I'm reattaching the JvmMetrics after the rm got transitioned to standby state. UgiMetrics is a different story since it fails when get reattached. As a quick fix I recreate it. Right now this patch is just demo for a quick fix. I think the root contradiction here is our metric system does not quite consider the case when a metric system got restarted (and metrics got reattached), while the RM HA code relaunches the metric system. If there are more elegant ways to fix this please do let me know. Thanks!

            People

            • Assignee:
              gtCarrera9 Li Lu
              Reporter:
              gtCarrera9 Li Lu
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development