HBase
  1. HBase
  2. HBASE-11007

BLOCKCACHE in schema descriptor seems not aptly named

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.94.19
    • Fix Version/s: 0.99.0
    • Component/s: None
    • Labels:
      None

      Description

      Hi,

      It seems that setting BLOCKCACHE key to false will disable the Data blocks from being cached but will continue to cache bloom and index blocks. This same property seems to be called cacheDataOnRead inside CacheConfig.

      Should this be called CACHE_DATA_ON_READ instead of BLOCKCACHE similar to the other CACHE_DATA_ON_WRITE/CACHE_INDEX_ON_WRITE. We got quite confused and ended up adding our own property CACHE_DATA_ON_READ - we also added some unit tests for the same.

      What do folks think about this ?

      Thanks
      Varun

      1. 11007v2.txt
        9 kB
        stack
      2. 11007.txt
        11 kB
        stack

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        30d 22h 23m 1 stack 17/May/14 22:28
        Patch Available Patch Available Resolved Resolved
        1d 18h 26m 1 stack 19/May/14 16:54
        Resolved Resolved Closed Closed
        278d 7h 35m 1 Enis Soztutar 21/Feb/15 23:30
        Enis Soztutar made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Enis Soztutar added a comment -

        Closing this issue after 0.99.0 release.

        Show
        Enis Soztutar added a comment - Closing this issue after 0.99.0 release.
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-TRUNK #5135 (See https://builds.apache.org/job/HBase-TRUNK/5135/)
        HBASE-11007 BLOCKCACHE in schema descriptor seems not aptly named (stack: rev 1595938)

        • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStats.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestForceCacheImportantBlocks.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK #5135 (See https://builds.apache.org/job/HBase-TRUNK/5135/ ) HBASE-11007 BLOCKCACHE in schema descriptor seems not aptly named (stack: rev 1595938) /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStats.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestForceCacheImportantBlocks.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-0.98 #309 (See https://builds.apache.org/job/HBase-0.98/309/)
        HBASE-11007 BLOCKCACHE in schema descriptor seems not aptly named; REVERT (stack: rev 1595942)

        • /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
        • /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
        • /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStats.java
        • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestForceCacheImportantBlocks.java
          HBASE-11007 BLOCKCACHE in schema descriptor seems not aptly named (stack: rev 1595939)
        • /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
        • /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
        • /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStats.java
        • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestForceCacheImportantBlocks.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-0.98 #309 (See https://builds.apache.org/job/HBase-0.98/309/ ) HBASE-11007 BLOCKCACHE in schema descriptor seems not aptly named; REVERT (stack: rev 1595942) /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStats.java /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestForceCacheImportantBlocks.java HBASE-11007 BLOCKCACHE in schema descriptor seems not aptly named (stack: rev 1595939) /hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStats.java /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestForceCacheImportantBlocks.java
        stack made changes -
        Fix Version/s 0.98.3 [ 12326765 ]
        Hide
        stack added a comment -

        Reverted from 0.98. Some issue in the test. Will work on it if wanted.

        Show
        stack added a comment - Reverted from 0.98. Some issue in the test. Will work on it if wanted.
        stack made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 0.98.3 [ 12326765 ]
        Resolution Fixed [ 1 ]
        Hide
        stack added a comment -

        Committed these javadoc and minor refactor of a test to 0.98 and to trunk.

        Show
        stack added a comment - Committed these javadoc and minor refactor of a test to 0.98 and to trunk.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12645436/11007v2.txt
        against trunk revision .
        ATTACHMENT ID: 12645436

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

        -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12645436/11007v2.txt against trunk revision . ATTACHMENT ID: 12645436 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. +1 javac . The applied patch does not increase the total number of javac compiler warnings. -1 javadoc . The javadoc tool appears to have generated 1 warning messages. -1 findbugs . The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9538//console This message is automatically generated.
        stack made changes -
        Attachment 11007v2.txt [ 12645436 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12645430/11007.txt
        against trunk revision .
        ATTACHMENT ID: 12645430

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9533//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12645430/11007.txt against trunk revision . ATTACHMENT ID: 12645430 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 6 new or modified tests. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9533//console This message is automatically generated.
        stack made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Affects Version/s 0.94.19 [ 12326287 ]
        Affects Version/s 0.94.18 [ 12325952 ]
        Fix Version/s 0.99.0 [ 12325675 ]
        Hide
        stack added a comment -

        Making the patch for trunk. I could make a patch for 0.94 to backport the javadoc piece (since it already has a test, a test that is better than what I'm adding to trunk). You want it Lars Hofhansl

        Show
        stack added a comment - Making the patch for trunk. I could make a patch for 0.94 to backport the javadoc piece (since it already has a test, a test that is better than what I'm adding to trunk). You want it Lars Hofhansl
        stack made changes -
        Field Original Value New Value
        Attachment 11007.txt [ 12645430 ]
        Hide
        stack added a comment -

        Changed javadoc for BLOCKCACHE and for its methods to explain that a better name would have been CACHE_DATA_ON_READ explaining that this attribute enables DATA block caching, yes/no.

        The TestForceCacheImportantBlocks in trunk was testing nothing (since removal of SchemaMetrics). Cache stats are opaque on whether DATA or META (INDEX/BLOOM). Let me fix that elsewhere. Meantime made TestForceCacheImportantBlocks do a very basic verification that when BLOCKCACHE is on/off, that the miss count reflects at least a difference.

        Show
        stack added a comment - Changed javadoc for BLOCKCACHE and for its methods to explain that a better name would have been CACHE_DATA_ON_READ explaining that this attribute enables DATA block caching, yes/no. The TestForceCacheImportantBlocks in trunk was testing nothing (since removal of SchemaMetrics). Cache stats are opaque on whether DATA or META (INDEX/BLOOM). Let me fix that elsewhere. Meantime made TestForceCacheImportantBlocks do a very basic verification that when BLOCKCACHE is on/off, that the miss count reflects at least a difference.
        Hide
        Nick Dimiduk added a comment -

        On 0.94, this test uses SchemaMetrics, which was removed as part of HBASE-6410.

        Show
        Nick Dimiduk added a comment - On 0.94, this test uses SchemaMetrics, which was removed as part of HBASE-6410 .
        Hide
        Lars Hofhansl added a comment -

        There's TestForceCacheImportantBlocks, which seems to test that... At least in 0.94.

        The test exists in trunk, however it looks like the test is not actually verifying anything anymore.
        stack, Andrew Purtell, FYI. Looks like the test should either be fixed to verify something or be removed.

        Show
        Lars Hofhansl added a comment - There's TestForceCacheImportantBlocks, which seems to test that... At least in 0.94. The test exists in trunk, however it looks like the test is not actually verifying anything anymore. stack , Andrew Purtell , FYI. Looks like the test should either be fixed to verify something or be removed.
        Hide
        Varun Sharma added a comment -

        I agree on the backward compatibility issue. How about at least adding a test, as part of this issue, on this property which exercises the code path and makes sure that it only touches Data Blocks ? I found several tests on CACHE_DATA_ON_WRITE etc. but none for this option ? Did I miss the unit test by any chance ?

        Thanks !
        Varun

        Show
        Varun Sharma added a comment - I agree on the backward compatibility issue. How about at least adding a test, as part of this issue, on this property which exercises the code path and makes sure that it only touches Data Blocks ? I found several tests on CACHE_DATA_ON_WRITE etc. but none for this option ? Did I miss the unit test by any chance ? Thanks ! Varun
        Hide
        Lars Hofhansl added a comment -

        Debatable. There aren't many index/bloomfilter blocks and not caching those would lead to truly terrible performance. Hence the only useful option is whether enable or disable caching of data blocks.
        We can't rename those without backward compatibility issues.

        Show
        Lars Hofhansl added a comment - Debatable. There aren't many index/bloomfilter blocks and not caching those would lead to truly terrible performance. Hence the only useful option is whether enable or disable caching of data blocks. We can't rename those without backward compatibility issues.
        Varun Sharma created issue -

          People

          • Assignee:
            Varun Sharma
            Reporter:
            Varun Sharma
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development