Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-1186

Memory-aware Maps with LRU eviction for Cell Cache

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.0
    • Component/s: None
    • Labels:
      None

      Description

      Caching is key for 0.20. We need a set of memory-aware data structures to manage our caches.

      I propose two initial classes: LruHashMap and LruBlockMap

      LruHashMap is currently being used over in HBASE-80 for the Cell cache. Erik Holstad has done extensive testing and benchmarking and will post results over in this issue.

      • Memory-aware
      • Fixed size
      • LRU eviction

      LruBlockMap can be used for the block caching of the new file format in HBASE-61. It should try to use all available memory, but must contend with Memcaches so is resizable to deal with heap pressure. Adding high priority blocks (evicted last) gives us in-memory functionality as described in bigtable paper.

      • Memory-aware
      • Fully resizable
      • LRU eviction (with some additions)
      • High priority blocks
      • Optional: Scan resistant algorithm

      Part of this issue is also solving how we will determine the size of cached objects.

        Attachments

        1. HeapSize.java
          2 kB
          Jonathan Gray
        2. LruHashMap.java
          30 kB
          Jonathan Gray
        3. hbase-1186-v2.patch
          35 kB
          Jonathan Gray
        4. hbase-1186-v3.patch
          33 kB
          Jonathan Gray
        5. hbase-1186-v4.patch
          45 kB
          Jonathan Gray

          Issue Links

            Activity

              People

              • Assignee:
                streamy Jonathan Gray
                Reporter:
                streamy Jonathan Gray
              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: