Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-8894

Forward port compressed l2 cache from 0.89fb

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Critical
    • Resolution: Implemented
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Forward port Alex's improvement on hbase-7407 from 0.89-fb branch:

        1 r1492797 | liyin | 2013-06-13 11:18:20 -0700 (Thu, 13 Jun 2013) | 43 lines
        2
        3 [master] Implements a secondary compressed cache (L2 cache)
        4
        5 Author: avf
        6
        7 Summary:
        8 This revision implements compressed and encoded second-level cache with off-heap
        9 (and optionally on-heap) storage and a bucket-allocator based on HBASE-7404.
       10
       11 BucketCache from HBASE-7404 is extensively modified to:
       12
       13 * Only handle byte arrays (i.e., no more serialization/deserialization within)
       14 * Remove persistence support for the time being
       15 * Keep an  index of hfilename to blocks for efficient eviction on close
       16
       17 A new interface (L2Cache) is introduced in order to separate it from the current
       18 implementation. The L2 cache is then integrated into the classes that handle
       19 reading from and writing to HFiles to allow cache-on-write as well as
       20 cache-on-read. Metrics for the L2 cache are integrated into RegionServerMetrics
       21 much in the same fashion as metrics for the existing (L2) BlockCache.
       22
       23 Additionally, CacheConfig class is re-refactored to configure the L2 cache,
       24 replace multile constructors with a Builder, as well as replace static methods
       25 for instantiating the caches with abstract factories (with singleton
       26 implementations for both the existing LruBlockCache and the newly introduced
       27 BucketCache based L2 cache)
       28
       29 Test Plan:
       30 1) Additional unit tests
       31 2) Stress test on a single devserver
       32 3) Test on a single-node in shadow cluster
       33 4) Test on a whole shadow cluster
       34
       35 Revert Plan:
       36
       37 Reviewers: liyintang, aaiyer, rshroff, manukranthk, adela
       38
       39 Reviewed By: liyintang
       40
       41 CC: gqchen, hbase-eng@
       42
       43 Differential Revision: https://phabricator.fb.com/D837264
       44
       45 Task ID: 2325295
       7 ------------------------------------------------------------------------
        6 r1492340 | liyin | 2013-06-12 11:36:03 -0700 (Wed, 12 Jun 2013) | 21 lines
      

        Attachments

        1. HBASE-8894-0.94-v1.txt
          211 kB
          Liang Xie
        2. HBASE-8894-0.94-v2.txt
          210 kB
          Liang Xie

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                stack Michael Stack
              • Votes:
                0 Vote for this issue
                Watchers:
                16 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: