[HBASE-1186] Memory-aware Maps with LRU eviction for Cell Cache

• Type: New Feature
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 0.20.0
• Assignee: Jonathan Gray
• Reporter: Jonathan Gray
• Created: 05/Feb/09 17:56
• Updated: 02/May/13 02:29
• Resolved: 26/Mar/09 01:47

Description

Caching is key for 0.20. We need a set of memory-aware data structures to manage our caches.

I propose two initial classes: LruHashMap and LruBlockMap

LruHashMap is currently being used over in HBASE-80 for the Cell cache. Erik Holstad has done extensive testing and benchmarking and will post results over in this issue.

• Memory-aware
• Fixed size
• LRU eviction

LruBlockMap can be used for the block caching of the new file format in HBASE-61. It should try to use all available memory, but must contend with Memcaches so is resizable to deal with heap pressure. Adding high priority blocks (evicted last) gives us in-memory functionality as described in bigtable paper.

• Memory-aware
• Fully resizable
• LRU eviction (with some additions)
• High priority blocks
• Optional: Scan resistant algorithm

Part of this issue is also solving how we will determine the size of cached objects.

Issue Links

• depends upon: HBASE-1188 Memory size of Java Objects - Make cacheable objects implement HeapSize
• is depended upon by: HBASE-80 [hbase] Add a cache of 'hot' cells