[HBASE-26274] Create an option to reintroduce BlockCache to mapreduce job

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha-1, 2.4.6
• Fix Version/s: 3.0.0-alpha-1, 2.5.0, 2.3.7, 2.4.7
• Component/s: BlockCache, HFile, mapreduce

Release Note

Introduce `hfile.onheap.block.cache.fixed.size` and default to disable. When using ClientSideRegionScanner, it will be enabled with a fixed size for caching INDEX/LEAF_INDEX block when a client, e.g. snapshot scanner, scans the entire HFile and does not need to seek/reseek to index block multiple times.

• Assignee: Tak-Lon (Stephen) Wu
• Reporter: Tak-Lon (Stephen) Wu
• Created: 09/Sep/21 18:49
• Updated: 24/Sep/21 02:42
• Resolved: 22/Sep/21 16:45

Description

In HBASE-21498 (see this commit, it change the behavior that only region server can initialize on-heap BlockCache/LruBlockCache, this should be the right change for HMaster.

Other downstream dependency that uses getScanner from a file-based region and read HStore/HFile lost the BlockCache for caching INDEX/LEAF_INDEX (at least still a problem with HBase-2.4) after this change (it worked before) and caused performance impact with 2x slower.

One way to bring back the performance is to allow non-RS and non-HMaster can use a compact version of blockcache with smaller memory and less hbase internal configuration.

Or if we can find a way to cache or skip reading the same LEAF_INDEX when scanning the DATA block with HFile.

Issue Links

• relates to: HBASE-26273 TableSnapshotInputFormat/TableSnapshotInputFormatImpl should use ReadType.STREAM for scanning HFiles
• links to: GitHub Pull Request #3684
• links to: GitHub Pull Request #3695