HBase
  1. HBase
  2. HBASE-11211

LoadTestTool option for specifying number of regions per server

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.99.0, 0.98.3
    • Component/s: None
    • Labels:
      None

      Description

      Add a new LoadTestTool option for specifying number of regions per server.

      1. HBASE-11211-trunk.patch
        4 kB
        Andrew Purtell
      2. HBASE-11211-0.98.patch
        4 kB
        Andrew Purtell

        Activity

        Hide
        Enis Soztutar added a comment -

        Closing this issue after 0.99.0 release.

        Show
        Enis Soztutar added a comment - Closing this issue after 0.99.0 release.
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-0.98 #312 (See https://builds.apache.org/job/HBase-0.98/312/)
        HBASE-11211 LoadTestTool option for specifying number of regions per server (apurtell: rev c93ceed290ed5b79c72dafa4252be0d5ca88ff79)

        • hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-0.98 #312 (See https://builds.apache.org/job/HBase-0.98/312/ ) HBASE-11211 LoadTestTool option for specifying number of regions per server (apurtell: rev c93ceed290ed5b79c72dafa4252be0d5ca88ff79) hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-TRUNK #5138 (See https://builds.apache.org/job/HBase-TRUNK/5138/)
        HBASE-11211 LoadTestTool option for specifying number of regions per server (apurtell: rev fd94fcde583c66e9f231342948c3e677a99a6d3b)

        • hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #5138 (See https://builds.apache.org/job/HBase-TRUNK/5138/ ) HBASE-11211 LoadTestTool option for specifying number of regions per server (apurtell: rev fd94fcde583c66e9f231342948c3e677a99a6d3b) hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java
        Hide
        Andrew Purtell added a comment -

        Committed to trunk and 0.98

        Show
        Andrew Purtell added a comment - Committed to trunk and 0.98
        Hide
        Andrew Purtell added a comment -

        Commit on hold until the SVN->GIT migration is finished.

        Show
        Andrew Purtell added a comment - Commit on hold until the SVN->GIT migration is finished.
        Hide
        Nick Dimiduk added a comment -

        Very well. No objection.

        Show
        Nick Dimiduk added a comment - Very well. No objection.
        Hide
        Andrew Purtell added a comment -

        Looks like a println snuck in.

        No that's intentional. LTT prints other variable configuration to stdout when starting. Let me know if you want me to remove this particular instance.

        Show
        Andrew Purtell added a comment - Looks like a println snuck in. No that's intentional. LTT prints other variable configuration to stdout when starting. Let me know if you want me to remove this particular instance.
        Hide
        Nick Dimiduk added a comment -

        Looks like a println snuck in.

        +    System.out.println("Regions per server: " + regionsPerServer);
        

        Otherwise, +1.

        Show
        Nick Dimiduk added a comment - Looks like a println snuck in. + System.out.println("Regions per server: " + regionsPerServer); Otherwise, +1.
        Hide
        Andrew Purtell added a comment -

        Going to commit trivial change to test code using CTR tonight unless objection.

        Show
        Andrew Purtell added a comment - Going to commit trivial change to test code using CTR tonight unless objection.
        Hide
        Andrew Purtell added a comment -

        Test failures definitely not related

        Show
        Andrew Purtell added a comment - Test failures definitely not related
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12645822/HBASE-11211-trunk.patch
        against trunk revision .
        ATTACHMENT ID: 12645822

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

        -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.client.TestHCM

        -1 core zombie tests. There are 1 zombie test(s): at org.apache.hadoop.hbase.mapreduce.TestMultiTableInputFormat.testScan(TestMultiTableInputFormat.java:244)
        at org.apache.hadoop.hbase.mapreduce.TestMultiTableInputFormat.testScanYZYToEmpty(TestMultiTableInputFormat.java:195)

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12645822/HBASE-11211-trunk.patch against trunk revision . ATTACHMENT ID: 12645822 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 6 new or modified tests. +1 javac . The applied patch does not increase the total number of javac compiler warnings. -1 javadoc . The javadoc tool appears to have generated 1 warning messages. -1 findbugs . The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.client.TestHCM -1 core zombie tests . There are 1 zombie test(s): at org.apache.hadoop.hbase.mapreduce.TestMultiTableInputFormat.testScan(TestMultiTableInputFormat.java:244) at org.apache.hadoop.hbase.mapreduce.TestMultiTableInputFormat.testScanYZYToEmpty(TestMultiTableInputFormat.java:195) Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9557//console This message is automatically generated.
        Hide
        Jean-Marc Spaggiari added a comment -

        Oh, hey, I see it now. DEFAULT_REGIONS_PER_SERVER was already 5. Just that now can be already adjusted from the command line.

        Thanks Andrew.

        Show
        Jean-Marc Spaggiari added a comment - Oh, hey, I see it now. DEFAULT_REGIONS_PER_SERVER was already 5. Just that now can be already adjusted from the command line. Thanks Andrew.
        Hide
        Andrew Purtell added a comment -

        Previously it was hard coded at 5 per RS

        Show
        Andrew Purtell added a comment - Previously it was hard coded at 5 per RS
        Hide
        Jean-Marc Spaggiari added a comment -

        LGTM. +1

        What's current number of regions created with today's code? Only 5? Or 5 like with this patch?

        Show
        Jean-Marc Spaggiari added a comment - LGTM. +1 What's current number of regions created with today's code? Only 5? Or 5 like with this patch?
        Hide
        Andrew Purtell added a comment -

        Attached patches for trunk and 0.98.

        Used the 0.98 patch just now to load up a regionserver with 1M regions.

        LoadTestTool uses HBaseTestingUtility to create tables so this change can benefit any user of HTU.

        Show
        Andrew Purtell added a comment - Attached patches for trunk and 0.98. Used the 0.98 patch just now to load up a regionserver with 1M regions. LoadTestTool uses HBaseTestingUtility to create tables so this change can benefit any user of HTU.

          People

          • Assignee:
            Andrew Purtell
            Reporter:
            Andrew Purtell
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development