Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-7498

Make REST server thread pool size configurable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.94.5, 0.95.2
    • Fix Version/s: 0.94.5, 0.95.0
    • Component/s: REST
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Added two configuration parameters hbase.rest.threads.max (default 100) and hbase.rest.threads.min (default 2) to configure the thread pool of a REST server.

      Before that, the thread pool is not configurable, and it has the default value max threads: 250, min threads: 2.
      Show
      Added two configuration parameters hbase.rest.threads.max (default 100) and hbase.rest.threads.min (default 2) to configure the thread pool of a REST server. Before that, the thread pool is not configurable, and it has the default value max threads: 250, min threads: 2.

      Description

      Currently, the REST server thread pool size is the default: 250. It can't be configured. We need to make it configurable so that it can be adjusted per traffic/load so that REST server is less likely to OOM and die.

      1. 0.94-7498_v2.patch
        3 kB
        Jimmy Xiang
      2. trunk-7498_2.patch
        3 kB
        Jimmy Xiang
      3. trunk-7498.patch
        1 kB
        Jimmy Xiang

        Issue Links

          Activity

          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12563373/trunk-7498.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 core tests. The patch passed unit tests in .

          -1 core zombie tests. There are 1 zombie test(s):

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3869//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3869//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3869//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3869//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3869//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3869//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3869//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3869//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3869//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12563373/trunk-7498.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 core tests . The patch passed unit tests in . -1 core zombie tests . There are 1 zombie test(s): Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3869//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3869//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3869//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3869//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3869//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3869//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3869//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3869//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3869//console This message is automatically generated.
          Hide
          jxiang Jimmy Xiang added a comment -

          I manually tested it and I can change the thread pool size with the patch.

          Show
          jxiang Jimmy Xiang added a comment - I manually tested it and I can change the thread pool size with the patch.
          Hide
          apurtell Andrew Purtell added a comment -

          Document these in hbase-defaults.xml and it's good to go. Thanks Jimmy.

          Show
          apurtell Andrew Purtell added a comment - Document these in hbase-defaults.xml and it's good to go. Thanks Jimmy.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12563384/trunk-7498_2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.regionserver.TestSplitTransaction

          -1 core zombie tests. There are 1 zombie test(s):

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3871//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3871//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3871//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3871//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3871//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3871//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3871//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3871//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3871//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12563384/trunk-7498_2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestSplitTransaction -1 core zombie tests . There are 1 zombie test(s): Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3871//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3871//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3871//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3871//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3871//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3871//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3871//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3871//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3871//console This message is automatically generated.
          Hide
          jxiang Jimmy Xiang added a comment -

          Integrated into trunk and 0.94. Thanks Andy for reviewing it.

          Show
          jxiang Jimmy Xiang added a comment - Integrated into trunk and 0.94. Thanks Andy for reviewing it.
          Hide
          hudson Hudson added a comment -

          Integrated in HBase-TRUNK #3701 (See https://builds.apache.org/job/HBase-TRUNK/3701/)
          HBASE-7498 Make REST server thread pool size configurable (Revision 1429364)

          Result = FAILURE
          jxiang :
          Files :

          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java
          • /hbase/trunk/hbase-server/src/main/resources/hbase-default.xml
          Show
          hudson Hudson added a comment - Integrated in HBase-TRUNK #3701 (See https://builds.apache.org/job/HBase-TRUNK/3701/ ) HBASE-7498 Make REST server thread pool size configurable (Revision 1429364) Result = FAILURE jxiang : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java /hbase/trunk/hbase-server/src/main/resources/hbase-default.xml
          Hide
          hudson Hudson added a comment -

          Integrated in HBase-0.94 #703 (See https://builds.apache.org/job/HBase-0.94/703/)
          HBASE-7498 Make REST server thread pool size configurable (Revision 1429365)

          Result = FAILURE
          jxiang :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/rest/Main.java
          • /hbase/branches/0.94/src/main/resources/hbase-default.xml
          Show
          hudson Hudson added a comment - Integrated in HBase-0.94 #703 (See https://builds.apache.org/job/HBase-0.94/703/ ) HBASE-7498 Make REST server thread pool size configurable (Revision 1429365) Result = FAILURE jxiang : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/rest/Main.java /hbase/branches/0.94/src/main/resources/hbase-default.xml
          Hide
          hudson Hudson added a comment -

          Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #331 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/331/)
          HBASE-7498 Make REST server thread pool size configurable (Revision 1429364)

          Result = FAILURE
          jxiang :
          Files :

          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java
          • /hbase/trunk/hbase-server/src/main/resources/hbase-default.xml
          Show
          hudson Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #331 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/331/ ) HBASE-7498 Make REST server thread pool size configurable (Revision 1429364) Result = FAILURE jxiang : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java /hbase/trunk/hbase-server/src/main/resources/hbase-default.xml
          Hide
          hudson Hudson added a comment -

          Integrated in HBase-0.94-security #95 (See https://builds.apache.org/job/HBase-0.94-security/95/)
          HBASE-7498 Make REST server thread pool size configurable (Revision 1429365)

          Result = SUCCESS
          jxiang :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/rest/Main.java
          • /hbase/branches/0.94/src/main/resources/hbase-default.xml
          Show
          hudson Hudson added a comment - Integrated in HBase-0.94-security #95 (See https://builds.apache.org/job/HBase-0.94-security/95/ ) HBASE-7498 Make REST server thread pool size configurable (Revision 1429365) Result = SUCCESS jxiang : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/rest/Main.java /hbase/branches/0.94/src/main/resources/hbase-default.xml
          Hide
          hudson Hudson added a comment -

          Integrated in HBase-0.94-security-on-Hadoop-23 #11 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/11/)
          HBASE-7498 Make REST server thread pool size configurable (Revision 1429365)

          Result = FAILURE
          jxiang :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/rest/Main.java
          • /hbase/branches/0.94/src/main/resources/hbase-default.xml
          Show
          hudson Hudson added a comment - Integrated in HBase-0.94-security-on-Hadoop-23 #11 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/11/ ) HBASE-7498 Make REST server thread pool size configurable (Revision 1429365) Result = FAILURE jxiang : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/rest/Main.java /hbase/branches/0.94/src/main/resources/hbase-default.xml

            People

            • Assignee:
              jxiang Jimmy Xiang
              Reporter:
              jxiang Jimmy Xiang
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development