Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-10403

Simplify offheap cache configuration

    XMLWordPrintableJSON

    Details

    • Type: Umbrella
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: io
    • Labels:
      None

      Description

      The BucketCache (HBASE-7404) is a very nice piece of functionality which is hidden behind complex configuration. Enabling it currently requires manual calculation of L1 cache. It'd be nice to make this easier to use and conform better with the existing heap management tools we already have.

      Turning it on currently requires explicitly setting LruBlockCache (L1) instance size and IOEngine (L2) size, making sure that L1 size isn't too big vs global memstore and total heap. This is further confused by hbase.bucketcache.size accepting a percentage of total heap OR explicit size in MB. Enabling SlabCache is slightly easier in that it just accepts whatever LruBlockCache is provided.

      Turning on BucketCache using off-heap mode could look like:

      hbase-env.sh:
      Set HBASE_REGIONSERVER_OPTS:
      -Xmx5000m
      -XX:MaxDirectMemorySize=15000m

      hbase-site.xml:

      • hbase.regionserver.global.memstore.size = 0.7
      • hbase.regionserver.onheap.blockcache.size = 0.1
      • hbase.regionserver.blockcache.impl = BucketCache
      • hbase.bucketcache.ioengine = offheap

      The result being a CombinedCache instance with 500m LruBlockCache + 15000m ByteBufferIOEngine running in direct mode.

      This example does a couple things (mostly for the admin):

      • knows NOT to enable SlabCache
      • s/hfile.block.cache.size/hbase.regionserver.onheap.blockcache.size/
      • maintains the validity of HBaseConfiguration's existing check that global MemStore + LruBlockCache == 0.8
      • maps "BucketCache" into meaning "a CombinedCache instance with these implementations for L1 and L2."
      • Figures out appropriate values for hbase.bucketcache.size and hbase.bucketcache.percentage.in.combinedcache

        Attachments

        1. HBASE-10403.0.patch
          35 kB
          Nick Dimiduk

          Issue Links

          There are no Sub-Tasks for this issue.

            Activity

              People

              • Assignee:
                ndimiduk Nick Dimiduk
                Reporter:
                ndimiduk Nick Dimiduk
              • Votes:
                0 Vote for this issue
                Watchers:
                14 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: