[HBASE-1460] Concurrent LRU Block Cache

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.20.0
• Component/s: io
• Hadoop Flags: Incompatible change, Reviewed
• Release Note: hfile.block.cache.size is not a percentage of total heap rather than absolute number
• Assignee: Jonathan Gray
• Reporter: Jonathan Gray
• Created: 31/May/09 00:29
• Updated: 13/Sep/09 22:24
• Resolved: 26/Jun/09 22:18

Description

The LRU-based block cache that will be committed in HBASE-1192 is thread-safe but contains a big lock on the hash map. Under high load, the block cache will be hit very heavily from a number of threads, so it needs to be built to handle massive concurrency.

This issue aims to implement a new block cache with LRU eviction, but backed by a ConcurrentHashMap and a separate eviction thread. Influence will be drawn from Solr's ConcurrentLRUCache, however there are major differences because solr treats all cached elements as equal size whereas we are dependent on our HeapSize interface with realistic (though approximate) heap usage.

Issue Links

• is depended upon by: HBASE-1218 Implement in-memory column