Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-1460

Concurrent LRU Block Cache

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.20.0
    • io
    • None
    • Incompatible change, Reviewed
    • hfile.block.cache.size is not a percentage of total heap rather than absolute number

    Description

      The LRU-based block cache that will be committed in HBASE-1192 is thread-safe but contains a big lock on the hash map. Under high load, the block cache will be hit very heavily from a number of threads, so it needs to be built to handle massive concurrency.

      This issue aims to implement a new block cache with LRU eviction, but backed by a ConcurrentHashMap and a separate eviction thread. Influence will be drawn from Solr's ConcurrentLRUCache, however there are major differences because solr treats all cached elements as equal size whereas we are dependent on our HeapSize interface with realistic (though approximate) heap usage.

      Attachments

        1. HBASE-1460-v5.patch
          90 kB
          Jonathan Gray
        2. HBASE-1460-v4.patch
          87 kB
          Jonathan Gray
        3. HBASE-1460-v3.patch
          79 kB
          Jonathan Gray
        4. HBASE-1460-v2.patch
          44 kB
          Jonathan Gray
        5. HBASE-1460-v1.patch
          35 kB
          Jonathan Gray

        Issue Links

          Activity

            People

              streamy Jonathan Gray
              streamy Jonathan Gray
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: