Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14200

Separate RegionReplica subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2

Details

    • Test
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 1.3.0, 2.0.0
    • None
    • None
    • Reviewed

    Description

      More and more functionality is added to StochasticLoadBalancer , making TestStochasticLoadBalancer run longer.

      From https://builds.apache.org/job/PreCommit-HBASE-Build/15011/testReport/org.apache.hadoop.hbase.master.balancer/TestStochasticLoadBalancer/ where total runtime was 14 min, here are the longest subtests:

      testRegionReplicasOnLargeCluster: 1 min 34 sec
      testRegionReplicasOnMidCluster: 1 min 31 sec
      testRegionReplicasOnMidClusterHighReplication: 2 min
      testRegionReplicationOnMidClusterReplicationGreaterThanNumNodes: 2 min 25 sec

      This issue is to separate out the above subtests into TestStochasticLoadBalancer2, giving each of the tests around 7 min runtime.

      Attachments

        1. 14200-v1.txt
          16 kB
          Ted Yu
        2. 14200-v2.txt
          9 kB
          Ted Yu

        Activity

          yuzhihong@gmail.com Ted Yu added a comment -

          From https://builds.apache.org/job/PreCommit-HBASE-Build/15014/console :

          Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 326.818 sec - in org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer
          ...
          Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 453.443 sec - in org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2
          
          yuzhihong@gmail.com Ted Yu added a comment - From https://builds.apache.org/job/PreCommit-HBASE-Build/15014/console : Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 326.818 sec - in org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer ... Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 453.443 sec - in org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12749427/14200-v1.txt
          against master branch at commit d4fc7cf2ba0d06b040dad435ab72366c4fd44116.
          ATTACHMENT ID: 12749427

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 5 new or modified tests.

          +1 hadoop versions. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 protoc. The applied patch does not increase the total number of protoc compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 checkstyle. The applied patch does not increase the total number of checkstyle errors

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces the following lines longer than 100:
          + int[] largeCluster = new int[] { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
          + RackManager rackManager, boolean assertFullyBalanced, boolean assertFullyBalancedForReplicas) {

          +1 site. The mvn post-site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/15014//testReport/
          Release Findbugs (version 2.0.3) warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/15014//artifact/patchprocess/newFindbugsWarnings.html
          Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/15014//artifact/patchprocess/checkstyle-aggregate.html

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/15014//console

          This message is automatically generated.

          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12749427/14200-v1.txt against master branch at commit d4fc7cf2ba0d06b040dad435ab72366c4fd44116. ATTACHMENT ID: 12749427 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 5 new or modified tests. +1 hadoop versions . The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0) +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 protoc . The applied patch does not increase the total number of protoc compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 checkstyle . The applied patch does not increase the total number of checkstyle errors +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces the following lines longer than 100: + int[] largeCluster = new int[] { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + RackManager rackManager, boolean assertFullyBalanced, boolean assertFullyBalancedForReplicas) { +1 site . The mvn post-site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/15014//testReport/ Release Findbugs (version 2.0.3) warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/15014//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/15014//artifact/patchprocess/checkstyle-aggregate.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/15014//console This message is automatically generated.
          jinghe Jerry He added a comment -

          After the move, the protected/private methods testWithCluster and createServerMap are duplicated in two test files?
          Is there any way to avoid duplicating the same code?

          jinghe Jerry He added a comment - After the move, the protected/private methods testWithCluster and createServerMap are duplicated in two test files? Is there any way to avoid duplicating the same code?
          yuzhihong@gmail.com Ted Yu added a comment -

          Patch v2 simplifies TestStochasticLoadBalancer2.java

          Please take a look.

          yuzhihong@gmail.com Ted Yu added a comment - Patch v2 simplifies TestStochasticLoadBalancer2.java Please take a look.
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12749564/14200-v2.txt
          against master branch at commit c7065c4c40e94bcce2035b8ea9813cfc6124a7e0.
          ATTACHMENT ID: 12749564

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 5 new or modified tests.

          +1 hadoop versions. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 protoc. The applied patch does not increase the total number of protoc compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 checkstyle. The applied patch does not increase the total number of checkstyle errors

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn post-site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/15028//testReport/
          Release Findbugs (version 2.0.3) warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/15028//artifact/patchprocess/newFindbugsWarnings.html
          Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/15028//artifact/patchprocess/checkstyle-aggregate.html

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/15028//console

          This message is automatically generated.

          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12749564/14200-v2.txt against master branch at commit c7065c4c40e94bcce2035b8ea9813cfc6124a7e0. ATTACHMENT ID: 12749564 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 5 new or modified tests. +1 hadoop versions . The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0) +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 protoc . The applied patch does not increase the total number of protoc compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 checkstyle . The applied patch does not increase the total number of checkstyle errors +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn post-site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/15028//testReport/ Release Findbugs (version 2.0.3) warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/15028//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/15028//artifact/patchprocess/checkstyle-aggregate.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/15028//console This message is automatically generated.
          anoop.hbase Anoop Sam John added a comment -

          LGTM

          anoop.hbase Anoop Sam John added a comment - LGTM
          jinghe Jerry He added a comment -

          +1

          jinghe Jerry He added a comment - +1
          yuzhihong@gmail.com Ted Yu added a comment -

          Thanks for the reviews, Anoop and Jerry.

          yuzhihong@gmail.com Ted Yu added a comment - Thanks for the reviews, Anoop and Jerry.
          hudson Hudson added a comment -

          SUCCESS: Integrated in HBase-1.3 #98 (See https://builds.apache.org/job/HBase-1.3/98/)
          HBASE-14200 Separate RegionReplica subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2 (tedyu: rev afa6fa6baa90e44cafed5e3eb226a9111de8f07a)

          • hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer2.java
          • hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java
          hudson Hudson added a comment - SUCCESS: Integrated in HBase-1.3 #98 (See https://builds.apache.org/job/HBase-1.3/98/ ) HBASE-14200 Separate RegionReplica subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2 (tedyu: rev afa6fa6baa90e44cafed5e3eb226a9111de8f07a) hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer2.java hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java
          hudson Hudson added a comment -

          FAILURE: Integrated in HBase-TRUNK #6710 (See https://builds.apache.org/job/HBase-TRUNK/6710/)
          HBASE-14200 Separate RegionReplica subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2 (tedyu: rev ce72ce998f2e9ad23329b48ab8e85912d642fef1)

          • hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer2.java
          • hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java
          hudson Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #6710 (See https://builds.apache.org/job/HBase-TRUNK/6710/ ) HBASE-14200 Separate RegionReplica subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2 (tedyu: rev ce72ce998f2e9ad23329b48ab8e85912d642fef1) hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer2.java hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java
          hudson Hudson added a comment -

          FAILURE: Integrated in HBase-1.3-IT #80 (See https://builds.apache.org/job/HBase-1.3-IT/80/)
          HBASE-14200 Separate RegionReplica subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2 (tedyu: rev afa6fa6baa90e44cafed5e3eb226a9111de8f07a)

          • hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer2.java
          • hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java
          hudson Hudson added a comment - FAILURE: Integrated in HBase-1.3-IT #80 (See https://builds.apache.org/job/HBase-1.3-IT/80/ ) HBASE-14200 Separate RegionReplica subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2 (tedyu: rev afa6fa6baa90e44cafed5e3eb226a9111de8f07a) hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer2.java hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java
          stack Michael Stack added a comment -

          Why not look at test and see if could run for shorter time to same effect rather than break it up into two pieces?

          stack Michael Stack added a comment - Why not look at test and see if could run for shorter time to same effect rather than break it up into two pieces?
          yuzhihong@gmail.com Ted Yu added a comment -

          Shortening tests in TestStochasticLoadBalancer[2] is on my mind.

          Haven't got time in implementing it though.

          yuzhihong@gmail.com Ted Yu added a comment - Shortening tests in TestStochasticLoadBalancer[2] is on my mind. Haven't got time in implementing it though.
          hudson Hudson added a comment -

          FAILURE: Integrated in HBase-1.2 #208 (See https://builds.apache.org/job/HBase-1.2/208/)
          HBASE-14200 Separate RegionReplica subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2 (eclark: rev c6bea07746582a6e743d098d2338ee9e611a90c0)

          • hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer2.java
          • hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java
          hudson Hudson added a comment - FAILURE: Integrated in HBase-1.2 #208 (See https://builds.apache.org/job/HBase-1.2/208/ ) HBASE-14200 Separate RegionReplica subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2 (eclark: rev c6bea07746582a6e743d098d2338ee9e611a90c0) hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer2.java hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java
          hudson Hudson added a comment -

          SUCCESS: Integrated in HBase-1.2-IT #175 (See https://builds.apache.org/job/HBase-1.2-IT/175/)
          HBASE-14200 Separate RegionReplica subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2 (eclark: rev c6bea07746582a6e743d098d2338ee9e611a90c0)

          • hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java
          • hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer2.java
          hudson Hudson added a comment - SUCCESS: Integrated in HBase-1.2-IT #175 (See https://builds.apache.org/job/HBase-1.2-IT/175/ ) HBASE-14200 Separate RegionReplica subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2 (eclark: rev c6bea07746582a6e743d098d2338ee9e611a90c0) hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer2.java

          People

            yuzhihong@gmail.com Ted Yu
            yuzhihong@gmail.com Ted Yu
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: