[HBASE-27743] Enhancements for the persistent cache

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Delivered
• Fix Version/s: 2.6.0
• Component/s: BucketCache
• Assignee: Shanmukha Haripriya Kota
• Reporter: Shanmukha Haripriya Kota
• Created: 22/Mar/23 18:06
• Updated: 1 week ago 11:11
• Resolved: 1 week ago 11:10

Description

Currently, we maintain a list of Hfiles on disk for which prefetch is complete and avoid prefetching those files after a restart. But we don't handle cases where blocks are evicted from the cache or when the region is moved and the blocks from that region are evicted (or not evicted). The Hfile is kept in the prefetch list forever and over time the prefetch list would have all the files of the cluster and the prefetch would never be executed. This change aims to update the prefetched list of HFiles upon block eviction.

 

Issue Links

• is related to: HBASE-28450 BuckeCache.evictBlocksByHfileName won't work after a cache recovery from file
• relates to: HBASE-28458 BucketCache.notifyFileCachingCompleted may incorrectly consider a file fully cached
• relates to: HBASE-27313 Persist list of Hfiles names for which prefetch is done
• relates to: HBASE-27686 Recovery of BucketCache and Prefetched data after RS Crash

Sub-Tasks

• Update the list of prefetched hfiles upon simple block eviction: Resolved
• Update the list of prefetched files upon region movement: Resolved
• Interrupt BucketCachePersister thread when BucketCache is shutdown: Resolved
• Define RPC API for cache cleaning: Resolved
• Tooling for parsing/reading the prefetch files list file: Resolved
• Persistent cache map can get corrupt if crash happens midway through the write: Resolved
• Put the cached time at the beginning of the block; run cache validation in the background when retrieving the persistent cache: Resolved