Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-965

Implement BlockCache using custom LRU/ARC mechanism rather than SoftReferences

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Minor
    • Resolution: Duplicate
    • 0.19.0
    • 0.20.0
    • None
    • None

    Description

      BlockCaching is now turned on by default as of HBASE-953. This improves all-around performance but is very taxing on the GC. Random reading tends to thrash the cache.

      As discussed, rather than rely on JVM LRU GC, we should implement our own tracking of cached blocks using an ARC-like algorithm. This will also allow users to set the cache size per-table or per-family rather than just taking from the allotted Heap pool.

      Paper on ARC algorithm (a scan-resistant LRU): http://www.almaden.ibm.com/cs/people/dmodha/ARC.pdf

      Attachments

        Issue Links

          is related to

          New Feature - A new feature of the product, which has yet to be developed. HBASE-1218 Implement in-memory column

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HBASE-288 Add in-memory caching of data

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Closed

          Task - A task that needs to be done. HBASE-953 Enable BLOCKCACHE by default [WAS -> Reevaluate HBASE-288 block caching work....?]

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            People

              Unassigned Unassigned
              streamy Jonathan Gray
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: