HBase
  1. HBase
  2. HBASE-11279

Block cache could be disabled by mistake

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.99.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      There is a weird test failure:

      java.lang.AssertionError
      	at org.junit.Assert.fail(Assert.java:86)
      	at org.junit.Assert.assertTrue(Assert.java:41)
      	at org.junit.Assert.assertTrue(Assert.java:52)
      	at org.apache.hadoop.hbase.io.hfile.TestPrefetch.readStoreFile(TestPrefetch.java:96)
      	at org.apache.hadoop.hbase.io.hfile.TestPrefetch.testPrefetch(TestPrefetch.java:66)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
      

      Looked into it, and found it could be because CacheConfig tries to compare a float with a long. Probably we should do this instead:

      -    if (cachePercentage == 0L) {
      +    if (cachePercentage <= 0.0001f) {
      
      1. hbase-11279.patch
        0.7 kB
        Jimmy Xiang

        Activity

        Hide
        Jimmy Xiang added a comment -

        Log shows 2014-05-31 14:49:16,075 INFO [pool-1-thread-1] hfile.CacheConfig(275): CacheConfig:disabled

        Show
        Jimmy Xiang added a comment - Log shows 2014-05-31 14:49:16,075 INFO [pool-1-thread-1] hfile.CacheConfig(275): CacheConfig:disabled
        Hide
        Nick Dimiduk added a comment -

        Strange. Do you know what the value of cacheConfig was?

        Show
        Nick Dimiduk added a comment - Strange. Do you know what the value of cacheConfig was?
        Hide
        Jimmy Xiang added a comment -

        I don't have the value of cacheConfig. The blockcache is null since there is a NPE:

        java.lang.NullPointerException
        	at org.apache.hadoop.hbase.io.hfile.TestPrefetch.readStoreFile(TestPrefetch.java:91)
        	at org.apache.hadoop.hbase.io.hfile.TestPrefetch.testPrefetch(TestPrefetch.java:66)
        	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:606)
        	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
        	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
        	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74
        

        The test works fine locally. But it fails in our jenkins. With the suggested change, it is ok both locally and in our jenkins.

        Show
        Jimmy Xiang added a comment - I don't have the value of cacheConfig. The blockcache is null since there is a NPE: java.lang.NullPointerException at org.apache.hadoop.hbase.io.hfile.TestPrefetch.readStoreFile(TestPrefetch.java:91) at org.apache.hadoop.hbase.io.hfile.TestPrefetch.testPrefetch(TestPrefetch.java:66) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74 The test works fine locally. But it fails in our jenkins. With the suggested change, it is ok both locally and in our jenkins.
        Hide
        stack added a comment -

        I thought we had it so it was not possible to disable block cache but looking at the code, there is nothing to prevent this. Patch looks fine to me.

        Show
        stack added a comment - I thought we had it so it was not possible to disable block cache but looking at the code, there is nothing to prevent this. Patch looks fine to me.
        Hide
        Nick Dimiduk added a comment -

        Yes, disabling entirely the BlockCache is still supported.

        Show
        Nick Dimiduk added a comment - Yes, disabling entirely the BlockCache is still supported.
        Hide
        Nick Dimiduk added a comment -

        Can't view the patch on mobile, but I'm good with what you propose, Jimmy.

        Show
        Nick Dimiduk added a comment - Can't view the patch on mobile, but I'm good with what you propose, Jimmy.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12647927/hbase-11279.patch
        against trunk revision .
        ATTACHMENT ID: 12647927

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        -1 core tests. The patch failed these unit tests:

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12647927/hbase-11279.patch against trunk revision . ATTACHMENT ID: 12647927 +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//console This message is automatically generated.
        Hide
        Jimmy Xiang added a comment -

        Thanks. I pushed in a slightly changed fix. But I realized the commit message was wrong.

        Show
        Jimmy Xiang added a comment - Thanks. I pushed in a slightly changed fix. But I realized the commit message was wrong.
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-TRUNK #5170 (See https://builds.apache.org/job/HBase-TRUNK/5170/)
        HBASE-11279 Block cache could be disabled by mistake (jxiang: rev 369141b79554f625752d0f6497e1426b3fa284e8)

        • hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestPrefetch.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerSelectionUsingKeyRange.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #5170 (See https://builds.apache.org/job/HBase-TRUNK/5170/ ) HBASE-11279 Block cache could be disabled by mistake (jxiang: rev 369141b79554f625752d0f6497e1426b3fa284e8) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestPrefetch.java hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerSelectionUsingKeyRange.java hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
        Hide
        Enis Soztutar added a comment -

        Closing this issue after 0.99.0 release.

        Show
        Enis Soztutar added a comment - Closing this issue after 0.99.0 release.

          People

          • Assignee:
            Jimmy Xiang
            Reporter:
            Jimmy Xiang
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development