[HBASE-4805] Allow better control of resource consumption in HTable

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.92.0, 0.94.0
• Fix Version/s: 0.92.0, 0.94.0
• Component/s: Client
• Hadoop Flags: Reviewed
• Assignee: Lars Hofhansl
• Reporter: Lars Hofhansl
• Created: 16/Nov/11 23:22
• Updated: 02/May/13 03:29
• Resolved: 18/Nov/11 00:15

Description

From some internal discussions at Salesforce we concluded that we need better control over the resources (mostly threads) consumed by HTable when used in a AppServer with many client threads.

Since HTable is not thread safe, the only options are cache them (in a custom thread local or using HTablePool) or to create them on-demand.

I propose a simple change: Add a new constructor to HTable that takes an optional ExecutorService and HConnection instance. That would make HTable a pretty lightweight object and we would manage the ES and HC separately.

I'll upload a patch a soon to get some feedback.

Issue Links

• is depended upon by: HBASE-6580 Deprecate HTablePool in favor of HConnection.getTable(...)
• relates to: HBASE-4956 Control direct memory buffer consumption by HBaseClient
• relates to: HBASE-4787 Make corePool as a configurable parameter in HTable
• relates to: HBASE-4970 Add a parameter so that keepAliveTime of Htable thread pool can be changed
• relates to: HBASE-4968 Add to troubleshooting workaround for direct buffer oome's.
• relates to: HBASE-5084 Allow different HTable instances to share one ExecutorService

Sub-Tasks

• Allow HBaseAdmin to use an existing connection: Closed
• Better HTable resource consumption in CoprocessorHost: Resolved