HBase
  1. HBase
  2. HBASE-5724

Row cache of KeyValue should be cleared in readFields().

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.92.1
    • Fix Version/s: 0.90.7, 0.92.2, 0.94.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      KeyValue does not clear its row cache in reading new values (readFields()).
      Therefore, If a KeyValue (kv) which caches its row bytes reads another KeyValue instance, kv.getRow() returns a wrong value.

      1. HBASE-5724.txt
        3 kB
        Teruyoshi Zenmyo
      2. HBASE-5724v2.txt
        3 kB
        stack
      3. 5724.092.txt
        2 kB
        stack

        Activity

        Hide
        Hudson added a comment -

        Integrated in HBase-0.92-security #105 (See https://builds.apache.org/job/HBase-0.92-security/105/)
        HBASE-5724 Row cache of KeyValue should be cleared in readFields() – add missing import (Revision 1310148)
        HBASE-5724 Row cache of KeyValue should be cleared in readFields() (Revision 1310068)

        Result = FAILURE
        stack :
        Files :

        • /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java

        stack :
        Files :

        • /hbase/branches/0.92/CHANGES.txt
        • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/KeyValue.java
        • /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
        Show
        Hudson added a comment - Integrated in HBase-0.92-security #105 (See https://builds.apache.org/job/HBase-0.92-security/105/ ) HBASE-5724 Row cache of KeyValue should be cleared in readFields() – add missing import (Revision 1310148) HBASE-5724 Row cache of KeyValue should be cleared in readFields() (Revision 1310068) Result = FAILURE stack : Files : /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java stack : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/KeyValue.java /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94-security #8 (See https://builds.apache.org/job/HBase-0.94-security/8/)
        HBASE-5724 Row cache of KeyValue should be cleared in readFields(). (Revision 1310067)

        Result = SUCCESS
        stack :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/KeyValue.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
        Show
        Hudson added a comment - Integrated in HBase-0.94-security #8 (See https://builds.apache.org/job/HBase-0.94-security/8/ ) HBASE-5724 Row cache of KeyValue should be cleared in readFields(). (Revision 1310067) Result = SUCCESS stack : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/KeyValue.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
        Hide
        Lars Hofhansl added a comment -

        Stack added missing import.

        Show
        Lars Hofhansl added a comment - Stack added missing import.
        Hide
        Hudson added a comment -

        Integrated in HBase-0.92 #358 (See https://builds.apache.org/job/HBase-0.92/358/)
        HBASE-5724 Row cache of KeyValue should be cleared in readFields() – add missing import (Revision 1310148)

        Result = FAILURE
        stack :
        Files :

        • /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
        Show
        Hudson added a comment - Integrated in HBase-0.92 #358 (See https://builds.apache.org/job/HBase-0.92/358/ ) HBASE-5724 Row cache of KeyValue should be cleared in readFields() – add missing import (Revision 1310148) Result = FAILURE stack : Files : /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
        Hide
        Lars Hofhansl added a comment -

        Strange, works for me locally in 0.90.

        Show
        Lars Hofhansl added a comment - Strange, works for me locally in 0.90.
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94 #90 (See https://builds.apache.org/job/HBase-0.94/90/)
        HBASE-5724 Row cache of KeyValue should be cleared in readFields(). (Revision 1310067)

        Result = FAILURE
        stack :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/KeyValue.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
        Show
        Hudson added a comment - Integrated in HBase-0.94 #90 (See https://builds.apache.org/job/HBase-0.94/90/ ) HBASE-5724 Row cache of KeyValue should be cleared in readFields(). (Revision 1310067) Result = FAILURE stack : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/KeyValue.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.92 #356 (See https://builds.apache.org/job/HBase-0.92/356/)
        HBASE-5724 Row cache of KeyValue should be cleared in readFields() (Revision 1310068)

        Result = FAILURE
        stack :
        Files :

        • /hbase/branches/0.92/CHANGES.txt
        • /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/KeyValue.java
        • /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
        Show
        Hudson added a comment - Integrated in HBase-0.92 #356 (See https://builds.apache.org/job/HBase-0.92/356/ ) HBASE-5724 Row cache of KeyValue should be cleared in readFields() (Revision 1310068) Result = FAILURE stack : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/KeyValue.java /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #2715 (See https://builds.apache.org/job/HBase-TRUNK/2715/)
        HBASE-5724 Row cache of KeyValue should be cleared in readFields(). (Revision 1310065)

        Result = FAILURE
        stack :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/KeyValue.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #2715 (See https://builds.apache.org/job/HBase-TRUNK/2715/ ) HBASE-5724 Row cache of KeyValue should be cleared in readFields(). (Revision 1310065) Result = FAILURE stack : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/KeyValue.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
        Hide
        Ted Yu added a comment -

        0.90 branch doesn't compile now:

        [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile (default-testCompile) on project hbase: Compilation failure
        [ERROR] /x1/jenkins/jenkins-slave/workspace/hbase-0.90/trunk/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java:[404,64] cannot find symbol
        [ERROR] symbol  : variable WritableUtils
        [ERROR] location: class org.apache.hadoop.hbase.TestKeyValue
        
        Show
        Ted Yu added a comment - 0.90 branch doesn't compile now: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile ( default -testCompile) on project hbase: Compilation failure [ERROR] /x1/jenkins/jenkins-slave/workspace/hbase-0.90/trunk/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java:[404,64] cannot find symbol [ERROR] symbol : variable WritableUtils [ERROR] location: class org.apache.hadoop.hbase.TestKeyValue
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-security #158 (See https://builds.apache.org/job/HBase-TRUNK-security/158/)
        HBASE-5724 Row cache of KeyValue should be cleared in readFields(). (Revision 1310065)

        Result = FAILURE
        stack :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/KeyValue.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-security #158 (See https://builds.apache.org/job/HBase-TRUNK-security/158/ ) HBASE-5724 Row cache of KeyValue should be cleared in readFields(). (Revision 1310065) Result = FAILURE stack : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/KeyValue.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
        Hide
        stack added a comment -

        Applied to 0.90, 0.92, 0.94 branches and trunk. Thanks for the patch Teruyoshi.

        Show
        stack added a comment - Applied to 0.90, 0.92, 0.94 branches and trunk. Thanks for the patch Teruyoshi.
        Hide
        stack added a comment -

        What I applied to 0.92 and 0.90

        Show
        stack added a comment - What I applied to 0.92 and 0.90
        Hide
        stack added a comment -

        Patch w/ comment amended as per Tedd's suggestion. This is what I am applying.

        Show
        stack added a comment - Patch w/ comment amended as per Tedd's suggestion. This is what I am applying.
        Hide
        Lars Hofhansl added a comment -

        +1 on 0.92 and 0.90
        This is a bad problem leading to hard track spurious problems.

        @Stack: Wanna do the commit, or should I?

        Show
        Lars Hofhansl added a comment - +1 on 0.92 and 0.90 This is a bad problem leading to hard track spurious problems. @Stack: Wanna do the commit, or should I?
        Hide
        Lars Hofhansl added a comment -

        TestFromClientSide passes locally with this patch applied (was worried that we made incorrect assumptions somewhere).

        Show
        Lars Hofhansl added a comment - TestFromClientSide passes locally with this patch applied (was worried that we made incorrect assumptions somewhere).
        Hide
        Todd Lipcon added a comment -

        Does this need to go back to 0.90.x as well?

        Show
        Todd Lipcon added a comment - Does this need to go back to 0.90.x as well?
        Hide
        stack added a comment -

        I can fix the comment on commit...

        Show
        stack added a comment - I can fix the comment on commit...
        Hide
        Lars Hofhansl added a comment -

        Good find +1

        Show
        Lars Hofhansl added a comment - Good find +1
        Hide
        stack added a comment -

        +1 on patch.

        Show
        stack added a comment - +1 on patch.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12521490/HBASE-5724.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.client.TestFromClientSide
        org.apache.hadoop.hbase.mapreduce.TestMultithreadedTableMapper
        org.apache.hadoop.hbase.mapreduce.TestImportTsv
        org.apache.hadoop.hbase.mapred.TestTableMapReduce
        org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
        org.apache.hadoop.hbase.mapreduce.TestTableMapReduce

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1403//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1403//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1403//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12521490/HBASE-5724.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestFromClientSide org.apache.hadoop.hbase.mapreduce.TestMultithreadedTableMapper org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat org.apache.hadoop.hbase.mapreduce.TestTableMapReduce Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1403//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1403//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1403//console This message is automatically generated.
        Hide
        Teruyoshi Zenmyo added a comment -

        The attached file is a patch which adds just one line together with a new test.

        • set rowcache of KeyValue to null in the readFields()
        • one new test which makes sure KeyValue.getRow() returns a correct value.
        Show
        Teruyoshi Zenmyo added a comment - The attached file is a patch which adds just one line together with a new test. set rowcache of KeyValue to null in the readFields() one new test which makes sure KeyValue.getRow() returns a correct value.

          People

          • Assignee:
            Teruyoshi Zenmyo
            Reporter:
            Teruyoshi Zenmyo
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development