[HBASE-1597] Prevent unnecessary caching of blocks during compactions

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.20.0
• Fix Version/s: 0.20.0
• Hadoop Flags: Reviewed
• Assignee: Jonathan Gray
• Reporter: Jonathan Gray
• Created: 01/Jul/09 05:01
• Updated: 10/Jun/11 00:37
• Resolved: 03/Jul/09 18:41

Description

When running any kind of compaction, we read every block of every storefile being compacted into the block cache.

We would like to reuse any already cached blocks, if available, but otherwise we should not bog down the LRU with unnecessary blocks.

This is not as bad as it was with the old LRU because the latest LRU implementation (HBASE-1460) is scan-resistant. This ensures that we are not causing massive eviction of the blocks that are being read multiple times or from in-memory tables. However, this does add to the GC-woes of an import because each block gets further referenced, and for longer periods of time. There is also overhead in running the LRU evictions.

Issue Links

• is related to: HBASE-3976 Disable Block Cache On Compactions