Hadoop YARN
  1. Hadoop YARN
  2. YARN-43

TestResourceTrackerService fail intermittently on jdk7

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.2-alpha, 0.23.3
    • Fix Version/s: 2.0.3-alpha, 0.23.5
    • Component/s: None
    • Labels:

      Description

      Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.73 sec <<< FAILURE!
      testDecommissionWithIncludeHosts(org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService) Time elapsed: 0.086 sec <<< FAILURE!
      junit.framework.AssertionFailedError: expected:<0> but was:<1> at junit.framework.Assert.fail(Assert.java:47)
      at junit.framework.Assert.failNotEquals(Assert.java:283)
      at junit.framework.Assert.assertEquals(Assert.java:64)
      at junit.framework.Assert.assertEquals(Assert.java:195)
      at junit.framework.Assert.assertEquals(Assert.java:201)
      at org.apache.hadoop.yarn.server.resourcemanager.TestResourceTrackerService.testDecommissionWithIncludeHosts(TestResourceTrackerService.java:90)

      1. YARN-43.patch
        1 kB
        Thomas Graves
      2. YARN-43.patch
        2 kB
        Thomas Graves

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1228 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1228/)
        YARN-43. Fix TestResourceTrackerService to not depend on test order and thus pass on JDK7. Contributed by Thomas Graves. (Revision 1399066)

        Result = FAILURE
        vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399066
        Files :

        • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1228 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1228/ ) YARN-43 . Fix TestResourceTrackerService to not depend on test order and thus pass on JDK7. Contributed by Thomas Graves. (Revision 1399066) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399066 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1198 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1198/)
        YARN-43. Fix TestResourceTrackerService to not depend on test order and thus pass on JDK7. Contributed by Thomas Graves. (Revision 1399066)

        Result = SUCCESS
        vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399066
        Files :

        • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1198 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1198/ ) YARN-43 . Fix TestResourceTrackerService to not depend on test order and thus pass on JDK7. Contributed by Thomas Graves. (Revision 1399066) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399066 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #407 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/407/)
        YARN-43. Fix TestResourceTrackerService to not depend on test order and thus pass on JDK7. Contributed by Thomas Graves.
        svn merge --ignore-ancestry -c 1399066 ../../trunk/ (Revision 1399069)

        Result = UNSTABLE
        vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399069
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-yarn-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java
        • /hadoop/common/branches/branch-0.23/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #407 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/407/ ) YARN-43 . Fix TestResourceTrackerService to not depend on test order and thus pass on JDK7. Contributed by Thomas Graves. svn merge --ignore-ancestry -c 1399066 ../../trunk/ (Revision 1399069) Result = UNSTABLE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399069 Files : /hadoop/common/branches/branch-0.23/hadoop-yarn-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java /hadoop/common/branches/branch-0.23/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Yarn-trunk #6 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/6/)
        YARN-43. Fix TestResourceTrackerService to not depend on test order and thus pass on JDK7. Contributed by Thomas Graves. (Revision 1399066)

        Result = FAILURE
        vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399066
        Files :

        • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
        Show
        Hudson added a comment - Integrated in Hadoop-Yarn-trunk #6 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/6/ ) YARN-43 . Fix TestResourceTrackerService to not depend on test order and thus pass on JDK7. Contributed by Thomas Graves. (Revision 1399066) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399066 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-trunk-Commit #2878 (See https://builds.apache.org/job/Hadoop-trunk-Commit/2878/)
        YARN-43. Fix TestResourceTrackerService to not depend on test order and thus pass on JDK7. Contributed by Thomas Graves. (Revision 1399066)

        Result = SUCCESS
        vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399066
        Files :

        • /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java
        • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
        Show
        Hudson added a comment - Integrated in Hadoop-trunk-Commit #2878 (See https://builds.apache.org/job/Hadoop-trunk-Commit/2878/ ) YARN-43 . Fix TestResourceTrackerService to not depend on test order and thus pass on JDK7. Contributed by Thomas Graves. (Revision 1399066) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399066 Files : /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClusterMetrics.java /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
        Hide
        Vinod Kumar Vavilapalli added a comment -

        Just committed this to trunk, branch-2 and branch-0.23. Thanks Thomas!

        Show
        Vinod Kumar Vavilapalli added a comment - Just committed this to trunk, branch-2 and branch-0.23. Thanks Thomas!
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12548567/YARN-43.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-YARN-Build/92//testReport/
        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/92//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12548567/YARN-43.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/92//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/92//console This message is automatically generated.
        Hide
        Vinod Kumar Vavilapalli added a comment -

        +1, looks good. Verified again that test-order doesn't affect the success of the tests. Kicking jenkins once more to be sure before committing.

        Show
        Vinod Kumar Vavilapalli added a comment - +1, looks good. Verified again that test-order doesn't affect the success of the tests. Kicking jenkins once more to be sure before committing.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12548567/YARN-43.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-YARN-Build/84//testReport/
        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/84//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12548567/YARN-43.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/84//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/84//console This message is automatically generated.
        Hide
        Thomas Graves added a comment -

        Changed this to have a destroy method in ClusterMetrics which gets called at teardown.

        I chose not to have shutdown method since RM shuts down the metrics system when it shuts down. I made this method visiblefortesting because I couldn't think of usecase for it to be public. If you can think of one let me know.

        Show
        Thomas Graves added a comment - Changed this to have a destroy method in ClusterMetrics which gets called at teardown. I chose not to have shutdown method since RM shuts down the metrics system when it shuts down. I made this method visiblefortesting because I couldn't think of usecase for it to be public. If you can think of one let me know.
        Hide
        Thomas Graves added a comment -

        Looks like the trunk version of the test changed the variable name. I'll look at implementing a destroy/reset type function in clusterMetrics.

        Show
        Thomas Graves added a comment - Looks like the trunk version of the test changed the variable name. I'll look at implementing a destroy/reset type function in clusterMetrics.
        Hide
        Vinod Kumar Vavilapalli added a comment -

        The patch doesn't compile code anymore.

        Also, instead of this, how about adding a shutdown/destroy method in ClusterMetrics and call it on every test's tearDown()? That way, each test-case can start with a clean slate. Thoughts?

        Show
        Vinod Kumar Vavilapalli added a comment - The patch doesn't compile code anymore. Also, instead of this, how about adding a shutdown/destroy method in ClusterMetrics and call it on every test's tearDown()? That way, each test-case can start with a clean slate. Thoughts?
        Hide
        Thomas Graves added a comment -

        The test was checking the metrics for decommissioned nodes against 0. Since with jdk7 the other tests in the file can be run first, the metrics had decommissioned nodes to 1 when testDecommissionWithIncludeHosts started.

        The test was already getting the initial metric count when starting the test, so simply change the assert to make sure the count hadn't changed instead of comparing to hardcoded 0.

        Show
        Thomas Graves added a comment - The test was checking the metrics for decommissioned nodes against 0. Since with jdk7 the other tests in the file can be run first, the metrics had decommissioned nodes to 1 when testDecommissionWithIncludeHosts started. The test was already getting the initial metric count when starting the test, so simply change the assert to make sure the count hadn't changed instead of comparing to hardcoded 0.

          People

          • Assignee:
            Thomas Graves
            Reporter:
            Thomas Graves
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development