[HBASE-22634] Improve performance of BufferedMutator

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Abandoned
• Affects Version/s: 2.1.5
• Component/s: Client

Environment

HDP 2.6.5

Linux RedHat

• Assignee: Unassigned
• Reporter: Sebastien Barnoud
• Created: 26/Jun/19 13:20
• Updated: 11/Jun/22 20:40
• Resolved: 11/Jun/22 20:40

Description

The default ThreadPoolExecutor uses a default size of 1 (property hbase.htable.threads.max). When using a size > 1, we still encountered poor performance and exception while submitting to the pool (pool exceed its capacity).

This patch propose a fix on different issues encountered when the pool size is > 1:

• thread safety issue
• concurrent cleanup by Netty and the "legacy" code
• errors in the backpressure
• Netty memory leak

And propose a BufferedMutatorThreadPoolExecutor which:

• uses hbase.client.max.total.tasks as the default size (instead of 1)
• some usefull metrics

 

Issue Links

• links to: GitHub Pull Request #343