Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-4805

Allow better control of resource consumption in HTable

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.92.0, 0.94.0
    • Fix Version/s: 0.92.0, 0.94.0
    • Component/s: Client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      From some internal discussions at Salesforce we concluded that we need better control over the resources (mostly threads) consumed by HTable when used in a AppServer with many client threads.

      Since HTable is not thread safe, the only options are cache them (in a custom thread local or using HTablePool) or to create them on-demand.

      I propose a simple change: Add a new constructor to HTable that takes an optional ExecutorService and HConnection instance. That would make HTable a pretty lightweight object and we would manage the ES and HC separately.

      I'll upload a patch a soon to get some feedback.

        Attachments

        1. 4805-v5.txt
          12 kB
          Lars Hofhansl
        2. 4805-v4.txt
          12 kB
          Lars Hofhansl
        3. 4805-v3.txt
          12 kB
          Lars Hofhansl
        4. 4805-v2.txt
          5 kB
          Lars Hofhansl
        5. 4805.txt
          5 kB
          Lars Hofhansl

          Issue Links

          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-6580 Deprecate HTablePool in favor of HConnection.getTable(...)

          • Major - Major loss of function.
          • Closed
          relates to

          New Feature - A new feature of the product, which has yet to be developed. HBASE-4956 Control direct memory buffer consumption by HBaseClient

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-4787 Make corePool as a configurable parameter in HTable

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-4970 Add a parameter so that keepAliveTime of Htable thread pool can be changed

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Closed

          Task - A task that needs to be done. HBASE-4968 Add to troubleshooting workaround for direct buffer oome's.

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. HBASE-5084 Allow different HTable instances to share one ExecutorService

          • Major - Major loss of function.
          • Closed
          There are no Sub-Tasks for this issue.

            Activity

              People

              • Assignee:
                larsh Lars Hofhansl
                Reporter:
                larsh Lars Hofhansl
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: