HBase
  1. HBase
  2. HBASE-11581

Add option so CombinedBlockCache L2 can be null (fscache)

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Invalid
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: BlockCache
    • Labels:

      Description

      Add option, mostly for comparison's sake, that allows a deploy orchestrated by CombinedBlockCache such that its L1 is LruBlockCache for META blocks but DATA blocks are fetched each time (we don't try and cache them, no blockcache churn).

      In operation, i can see fscache coming around to cover the fetched DATA blocks such that if the DATA blocks fit in fscache, seeks go to zero.

      This setup for sure runs slower. Will publish numbers elsewhere. Meantime, here is a patch to enable this option.

        Activity

        Hide
        Jonathan Hsieh added a comment -

        looks fine. Is the idea just for specialized benchmarking?

        Is there an easy way to get unit tests to exercise this new config?

        Show
        Jonathan Hsieh added a comment - looks fine. Is the idea just for specialized benchmarking? Is there an easy way to get unit tests to exercise this new config?
        Hide
        stack added a comment -

        Thanks Jonathan Hsieh for review.

        A better way to do this, a more 'natural' way is fixing the 'Cache DATA on Read' flag. Currently it is overloaded, compounded with the BLOCKCACHE => true/false flag. It needs to be made distinct. I just tried on a table setting CF to false but BC got deployed anyways.

        hbase(main):004:0> enable 'TestTable'
        0 row(s) in 0.8470 seconds
        
        hbase(main):005:0> describe 'TestTable'
        DESCRIPTION                                                                                                                         ENABLED
         'TestTable', {NAME => 'info', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '2', COMP true
         RESSION => 'NONE', MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_MEMORY => 'false
         ', BLOCKCACHE => 'false'}
        1 row(s) in 0.0430 seconds
        
        hbase(main):006:0>
        hbase(main):007:0*
        hbase(main):008:0* disable 'TestTable'
        0 row(s) in 1.2930 seconds
        
        hbase(main):009:0> describe 'TestTable'
        DESCRIPTION                                                                                                                         ENABLED
         'TestTable', {NAME => 'info', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '2', COMP false
         RESSION => 'NONE', MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_MEMORY => 'false
         ', BLOCKCACHE => 'false'}
        1 row(s) in 0.0340 seconds
        
        hbase(main):010:0> alter 'TestTable', {NAME => 'info', BLOCKCACHE => false}
        Updating all regions with the new schema...
        16/16 regions updated.
        Done.
        0 row(s) in 1.1950 seconds
        
        hbase(main):011:0> describe 'TestTable'
        DESCRIPTION                                                                                                                         ENABLED
         'TestTable', {NAME => 'info', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '2', COMP false
         RESSION => 'NONE', MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_MEMORY => 'false
         ', BLOCKCACHE => 'false'}
        1 row(s) in 0.0330 seconds
        
        hbase(main):012:0> enable 'TestTable'
        0 row(s) in 0.7850 seconds
        
        
        Show
        stack added a comment - Thanks Jonathan Hsieh for review. A better way to do this, a more 'natural' way is fixing the 'Cache DATA on Read' flag. Currently it is overloaded, compounded with the BLOCKCACHE => true/false flag. It needs to be made distinct. I just tried on a table setting CF to false but BC got deployed anyways. hbase(main):004:0> enable 'TestTable' 0 row(s) in 0.8470 seconds hbase(main):005:0> describe 'TestTable' DESCRIPTION ENABLED 'TestTable', {NAME => 'info', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '2', COMP true RESSION => 'NONE', MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => ' false ', BLOCKSIZE => '65536', IN_MEMORY => ' false ', BLOCKCACHE => ' false '} 1 row(s) in 0.0430 seconds hbase(main):006:0> hbase(main):007:0* hbase(main):008:0* disable 'TestTable' 0 row(s) in 1.2930 seconds hbase(main):009:0> describe 'TestTable' DESCRIPTION ENABLED 'TestTable', {NAME => 'info', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '2', COMP false RESSION => 'NONE', MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => ' false ', BLOCKSIZE => '65536', IN_MEMORY => ' false ', BLOCKCACHE => ' false '} 1 row(s) in 0.0340 seconds hbase(main):010:0> alter 'TestTable', {NAME => 'info', BLOCKCACHE => false } Updating all regions with the new schema... 16/16 regions updated. Done. 0 row(s) in 1.1950 seconds hbase(main):011:0> describe 'TestTable' DESCRIPTION ENABLED 'TestTable', {NAME => 'info', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '2', COMP false RESSION => 'NONE', MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => ' false ', BLOCKSIZE => '65536', IN_MEMORY => ' false ', BLOCKCACHE => ' false '} 1 row(s) in 0.0330 seconds hbase(main):012:0> enable 'TestTable' 0 row(s) in 0.7850 seconds
        Hide
        stack added a comment -

        Talat UYARER What you think of this one sir?

        Show
        stack added a comment - Talat UYARER What you think of this one sir?
        Hide
        stack added a comment -

        Resolving as won't fix. I can get the desired effect – caching of META blocks only and NOT caching DATA blocks – by setting BLOCKCACHE => 'false' (since HBASE-4683 "Always cache index and bloom blocks"). Testing I was confused by metrics showing DATA blocks count but the number was a persist from before the configuration change (Need to look into why metric not reset....that is another issue).

        Show
        stack added a comment - Resolving as won't fix. I can get the desired effect – caching of META blocks only and NOT caching DATA blocks – by setting BLOCKCACHE => 'false' (since HBASE-4683 "Always cache index and bloom blocks"). Testing I was confused by metrics showing DATA blocks count but the number was a persist from before the configuration change (Need to look into why metric not reset....that is another issue).

          People

          • Assignee:
            stack
            Reporter:
            stack
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development