[HBASE-27002] Config BucketCache as victim handler of LRUCache

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: BlockCache, BucketCache
• Labels: pull-request-available
• Assignee: Xuesen Liang
• Reporter: Xuesen Liang
• Created: 06/May/22 06:35
• Updated: 11/Apr/24 02:37

Description

Bucket cache is no longer as a victim handler for LRU by HBASE-19357.

When BC is used, data blocks will be strictly on BC only where as index/bloom blocks are on LRU L1 cache.

In this situation, the LRU cache's memory is totally on heap.

If the index and bloom filters on a region server are big, then a big LRU cache will introduce more GC cost.

So we should add a configuration for user to choose victim handler.

A small LRU cache with a big victim offheap bucket cache can reduce GC cost.

 

For example: a region server has 5GB index, 80 GB bloom filter, and 256GB DRAM, its configurations can be as follow:

export HBASE_REGIONSERVER_OPTS="-Xms40g -Xmx40g -XX:MaxDirectMemorySize=180g"

<property>
    <name>hbase.blockcache.victim.handler.enabled</name>
    <value>true</value>
</property>  

<property>
    <name>hfile.block.cache.size</name>
    <value>0.3</value>
</property> 

<property>
    <name>hbase.bucketcache.ioengine</name>
    <value>offheap</value>
</property>
<property>
    <name>hbase.bucketcache.size</name>
    <value>160000</value>
</property>

Issue Links

• links to: GitHub Pull Request #4410