[HBASE-28839] Exception handling during retrieval of bucket-cache from persistence.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-beta-1, 2.7.0
• Fix Version/s: 3.0.0, 2.7.0
• Component/s: BucketCache
• Labels: pull-request-available
• Assignee: Janardhan Hungund
• Reporter: Janardhan Hungund
• Created: 16/Sep/24 04:41
• Updated: 25/Sep/24 07:20
• Resolved: 24/Sep/24 11:57

Description

During the retrieval of bucket cache from the persistence file during the startup, it was observed that, if an exception, other than, the IOException occurs, the bucket cache internal members remain uninitialised and cause the bucket to remain unusable. The exception is not logged in the trace file and the retrieval thread exits without initialising the bucket-cache.

Also, the NullPointerExceptions are seen when, trying to use the cache.

2024-09-10 14:33:30,020 ERROR org.apache.hadoop.hbase.io.hfile.bucket.BucketCache: WriterThread encountered error
java.lang.NullPointerException
        at org.apache.hadoop.hbase.io.hfile.bucket.BucketCache$RAMQueueEntry.writeToCache(BucketCache.java:1975)
        at org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.doDrain(BucketCache.java:1298) 

 

2024-09-13 07:01:05,964 ERROR org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: Error getting metrics from source RegionServer,sub=Server
java.lang.NullPointerException
        at org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.getFreeSize(BucketCache.java:1819)
        at org.apache.hadoop.hbase.io.hfile.CombinedBlockCache.getFreeSize(CombinedBlockCache.java:179)
        at org.apache.hadoop.hbase.regionserver.MetricsRegionServerWrapperImpl.getBlockCacheFreeSize(MetricsRegionServerWrapperImpl.java:308)
        at org.apache.hadoop.hbase.regionserver.MetricsRegionServerSourceImpl.addGaugesToMetricsRecordBuilder(MetricsRegionServerSourceImpl.java:525)
        at org.apache.hadoop.hbase.regionserver.MetricsRegionServerSourceImpl.getMetrics(MetricsRegionServerSourceImpl.java:333) 

All type of exceptions need to be handled gracefully.
All types of exceptions must be logged to the trace file.
The bucket cache needs to reinitialised and made usable.

Thanks,
Janardhan

Issue Links

• is related to: HBASE-28805 Implement chunked persistence of backing map for persistent bucket cache.
• links to: GitHub Pull Request #6250
• links to: GitHub Pull Request #6288