Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-1186

Memory-aware Maps with LRU eviction for Cell Cache

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • None
    • 0.20.0
    • None
    • None

    Description

      Caching is key for 0.20. We need a set of memory-aware data structures to manage our caches.

      I propose two initial classes: LruHashMap and LruBlockMap

      LruHashMap is currently being used over in HBASE-80 for the Cell cache. Erik Holstad has done extensive testing and benchmarking and will post results over in this issue.

      • Memory-aware
      • Fixed size
      • LRU eviction

      LruBlockMap can be used for the block caching of the new file format in HBASE-61. It should try to use all available memory, but must contend with Memcaches so is resizable to deal with heap pressure. Adding high priority blocks (evicted last) gives us in-memory functionality as described in bigtable paper.

      • Memory-aware
      • Fully resizable
      • LRU eviction (with some additions)
      • High priority blocks
      • Optional: Scan resistant algorithm

      Part of this issue is also solving how we will determine the size of cached objects.

      Attachments

        1. hbase-1186-v2.patch
          35 kB
          Jonathan Gray
        2. hbase-1186-v3.patch
          33 kB
          Jonathan Gray
        3. hbase-1186-v4.patch
          45 kB
          Jonathan Gray
        4. HeapSize.java
          2 kB
          Jonathan Gray
        5. LruHashMap.java
          30 kB
          Jonathan Gray

        Issue Links

          Activity

            People

              streamy Jonathan Gray
              streamy Jonathan Gray
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: