HBase
  1. HBase
  2. HBASE-1186

Memory-aware Maps with LRU eviction for Cell Cache

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.0
    • Component/s: None
    • Labels:
      None

      Description

      Caching is key for 0.20. We need a set of memory-aware data structures to manage our caches.

      I propose two initial classes: LruHashMap and LruBlockMap

      LruHashMap is currently being used over in HBASE-80 for the Cell cache. Erik Holstad has done extensive testing and benchmarking and will post results over in this issue.

      • Memory-aware
      • Fixed size
      • LRU eviction

      LruBlockMap can be used for the block caching of the new file format in HBASE-61. It should try to use all available memory, but must contend with Memcaches so is resizable to deal with heap pressure. Adding high priority blocks (evicted last) gives us in-memory functionality as described in bigtable paper.

      • Memory-aware
      • Fully resizable
      • LRU eviction (with some additions)
      • High priority blocks
      • Optional: Scan resistant algorithm

      Part of this issue is also solving how we will determine the size of cached objects.

      1. hbase-1186-v2.patch
        35 kB
        Jonathan Gray
      2. hbase-1186-v3.patch
        33 kB
        Jonathan Gray
      3. hbase-1186-v4.patch
        45 kB
        Jonathan Gray
      4. HeapSize.java
        2 kB
        Jonathan Gray
      5. LruHashMap.java
        30 kB
        Jonathan Gray

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Jonathan Gray
              Reporter:
              Jonathan Gray
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development