HBase
  1. HBase
  2. HBASE-11279

Block cache could be disabled by mistake

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.99.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      There is a weird test failure:

      java.lang.AssertionError
      	at org.junit.Assert.fail(Assert.java:86)
      	at org.junit.Assert.assertTrue(Assert.java:41)
      	at org.junit.Assert.assertTrue(Assert.java:52)
      	at org.apache.hadoop.hbase.io.hfile.TestPrefetch.readStoreFile(TestPrefetch.java:96)
      	at org.apache.hadoop.hbase.io.hfile.TestPrefetch.testPrefetch(TestPrefetch.java:66)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
      

      Looked into it, and found it could be because CacheConfig tries to compare a float with a long. Probably we should do this instead:

      -    if (cachePercentage == 0L) {
      +    if (cachePercentage <= 0.0001f) {
      
      1. hbase-11279.patch
        0.7 kB
        Jimmy Xiang

        Activity

        Jimmy Xiang created issue -
        Hide
        Jimmy Xiang added a comment -

        Log shows 2014-05-31 14:49:16,075 INFO [pool-1-thread-1] hfile.CacheConfig(275): CacheConfig:disabled

        Show
        Jimmy Xiang added a comment - Log shows 2014-05-31 14:49:16,075 INFO [pool-1-thread-1] hfile.CacheConfig(275): CacheConfig:disabled
        Hide
        Nick Dimiduk added a comment -

        Strange. Do you know what the value of cacheConfig was?

        Show
        Nick Dimiduk added a comment - Strange. Do you know what the value of cacheConfig was?
        Hide
        Jimmy Xiang added a comment -

        I don't have the value of cacheConfig. The blockcache is null since there is a NPE:

        java.lang.NullPointerException
        	at org.apache.hadoop.hbase.io.hfile.TestPrefetch.readStoreFile(TestPrefetch.java:91)
        	at org.apache.hadoop.hbase.io.hfile.TestPrefetch.testPrefetch(TestPrefetch.java:66)
        	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:606)
        	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
        	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
        	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74
        

        The test works fine locally. But it fails in our jenkins. With the suggested change, it is ok both locally and in our jenkins.

        Show
        Jimmy Xiang added a comment - I don't have the value of cacheConfig. The blockcache is null since there is a NPE: java.lang.NullPointerException at org.apache.hadoop.hbase.io.hfile.TestPrefetch.readStoreFile(TestPrefetch.java:91) at org.apache.hadoop.hbase.io.hfile.TestPrefetch.testPrefetch(TestPrefetch.java:66) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74 The test works fine locally. But it fails in our jenkins. With the suggested change, it is ok both locally and in our jenkins.
        Jimmy Xiang made changes -
        Field Original Value New Value
        Attachment hbase-11279.patch [ 12647927 ]
        Jimmy Xiang made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Jimmy Xiang made changes -
        Priority Major [ 3 ] Minor [ 4 ]
        Hide
        stack added a comment -

        I thought we had it so it was not possible to disable block cache but looking at the code, there is nothing to prevent this. Patch looks fine to me.

        Show
        stack added a comment - I thought we had it so it was not possible to disable block cache but looking at the code, there is nothing to prevent this. Patch looks fine to me.
        Hide
        Nick Dimiduk added a comment -

        Yes, disabling entirely the BlockCache is still supported.

        Show
        Nick Dimiduk added a comment - Yes, disabling entirely the BlockCache is still supported.
        Hide
        Nick Dimiduk added a comment -

        Can't view the patch on mobile, but I'm good with what you propose, Jimmy.

        Show
        Nick Dimiduk added a comment - Can't view the patch on mobile, but I'm good with what you propose, Jimmy.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12647927/hbase-11279.patch
        against trunk revision .
        ATTACHMENT ID: 12647927

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        -1 core tests. The patch failed these unit tests:

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12647927/hbase-11279.patch against trunk revision . ATTACHMENT ID: 12647927 +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9665//console This message is automatically generated.
        Hide
        Jimmy Xiang added a comment -

        Thanks. I pushed in a slightly changed fix. But I realized the commit message was wrong.

        Show
        Jimmy Xiang added a comment - Thanks. I pushed in a slightly changed fix. But I realized the commit message was wrong.
        Jimmy Xiang made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Fix Version/s 0.99.0 [ 12325675 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-TRUNK #5170 (See https://builds.apache.org/job/HBase-TRUNK/5170/)
        HBASE-11279 Block cache could be disabled by mistake (jxiang: rev 369141b79554f625752d0f6497e1426b3fa284e8)

        • hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestPrefetch.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerSelectionUsingKeyRange.java
        • hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #5170 (See https://builds.apache.org/job/HBase-TRUNK/5170/ ) HBASE-11279 Block cache could be disabled by mistake (jxiang: rev 369141b79554f625752d0f6497e1426b3fa284e8) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestPrefetch.java hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerSelectionUsingKeyRange.java hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
        Hide
        Enis Soztutar added a comment -

        Closing this issue after 0.99.0 release.

        Show
        Enis Soztutar added a comment - Closing this issue after 0.99.0 release.
        Enis Soztutar made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        1d 12h 26m 1 Jimmy Xiang 02/Jun/14 17:42
        Patch Available Patch Available Resolved Resolved
        7h 32m 1 Jimmy Xiang 03/Jun/14 01:15
        Resolved Resolved Closed Closed
        263d 23h 18m 1 Enis Soztutar 21/Feb/15 23:33

          People

          • Assignee:
            Jimmy Xiang
            Reporter:
            Jimmy Xiang
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development