HBase
  1. HBase
  2. HBASE-5736

ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.94.0, 0.95.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We have fixed similar bug in
      https://issues.apache.org/jira/browse/HBASE-5507

      It uses ByteBuffer.array() to read the ByteBuffer.
      This will ignore the offset return the whole underlying byte array.
      The bug can be triggered by using framed Transport thrift servers.

        Activity

        Hide
        Phabricator added a comment -

        sc requested code review of "HBASE-5736 [jira] ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly".
        Reviewers: tedyu, dhruba, JIRA

        ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly

        Task ID: #

        Blame Rev:

        We have fixed similar bug in
        https://issues.apache.org/jira/browse/HBASE-5507

        It uses ByteBuffer.array() to read the ByteBuffer.
        This will ignore the offset return the whole underlying byte array.
        The bug can be triggered by using framed Transport thrift servers.

        TEST PLAN
        unit test

        Revert Plan:

        Tags:

        REVISION DETAIL
        https://reviews.facebook.net/D2649

        AFFECTED FILES
        src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
        src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
        src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java

        MANAGE HERALD DIFFERENTIAL RULES
        https://reviews.facebook.net/herald/view/differential/

        WHY DID I GET THIS EMAIL?
        https://reviews.facebook.net/herald/transcript/6087/

        Tip: use the X-Herald-Rules header to filter Herald messages in your client.

        Show
        Phabricator added a comment - sc requested code review of " HBASE-5736 [jira] ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly". Reviewers: tedyu, dhruba, JIRA ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly Task ID: # Blame Rev: We have fixed similar bug in https://issues.apache.org/jira/browse/HBASE-5507 It uses ByteBuffer.array() to read the ByteBuffer. This will ignore the offset return the whole underlying byte array. The bug can be triggered by using framed Transport thrift servers. TEST PLAN unit test Revert Plan: Tags: REVISION DETAIL https://reviews.facebook.net/D2649 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/6087/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
        Hide
        Phabricator added a comment -

        sc updated the revision "HBASE-5736 [jira] ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly".
        Reviewers: tedyu, dhruba, JIRA

        Remove some testing codes

        REVISION DETAIL
        https://reviews.facebook.net/D2649

        AFFECTED FILES
        src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
        src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
        src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java

        Show
        Phabricator added a comment - sc updated the revision " HBASE-5736 [jira] ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly". Reviewers: tedyu, dhruba, JIRA Remove some testing codes REVISION DETAIL https://reviews.facebook.net/D2649 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
        Hide
        Phabricator added a comment -

        tedyu has accepted the revision "HBASE-5736 [jira] ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly".

        If tests pass.

        REVISION DETAIL
        https://reviews.facebook.net/D2649

        BRANCH
        get-bytes-thrift

        Show
        Phabricator added a comment - tedyu has accepted the revision " HBASE-5736 [jira] ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly". If tests pass. REVISION DETAIL https://reviews.facebook.net/D2649 BRANCH get-bytes-thrift
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12521632/HBASE-5736.D2649.2.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
        org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1423//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1423//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1423//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12521632/HBASE-5736.D2649.2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1423//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1423//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1423//console This message is automatically generated.
        Hide
        Scott Chen added a comment -

        Failed test seems related. I will take a look.

        Show
        Scott Chen added a comment - Failed test seems related. I will take a look.
        Hide
        Scott Chen added a comment -

        The test got timeout. It makes sense because I am adding more steps to the test.
        Increasing the timeout should fix this.

        Show
        Scott Chen added a comment - The test got timeout. It makes sense because I am adding more steps to the test. Increasing the timeout should fix this.
        Hide
        Phabricator added a comment -

        sc updated the revision "HBASE-5736 [jira] ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly".
        Reviewers: tedyu, dhruba, JIRA

        Increase test timeout because more tests are added.

        REVISION DETAIL
        https://reviews.facebook.net/D2649

        AFFECTED FILES
        src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
        src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
        src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java

        Show
        Phabricator added a comment - sc updated the revision " HBASE-5736 [jira] ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly". Reviewers: tedyu, dhruba, JIRA Increase test timeout because more tests are added. REVISION DETAIL https://reviews.facebook.net/D2649 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12521707/HBASE-5736.D2649.3.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1432//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1432//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1432//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12521707/HBASE-5736.D2649.3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1432//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1432//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1432//console This message is automatically generated.
        Hide
        Ted Yu added a comment -

        The test failure was caused by OutOfMemoryError.

        Will integrate later if there is no objection.

        Show
        Ted Yu added a comment - The test failure was caused by OutOfMemoryError. Will integrate later if there is no objection.
        Hide
        Ted Yu added a comment -

        Integrated to trunk.

        Thanks for the patch, Scott.

        Show
        Ted Yu added a comment - Integrated to trunk. Thanks for the patch, Scott.
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #2725 (See https://builds.apache.org/job/HBase-TRUNK/2725/)
        HBASE-5736 ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly (Scott Chen) (Revision 1310634)

        Result = SUCCESS
        tedyu :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #2725 (See https://builds.apache.org/job/HBase-TRUNK/2725/ ) HBASE-5736 ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly (Scott Chen) (Revision 1310634) Result = SUCCESS tedyu : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94 #95 (See https://builds.apache.org/job/HBase-0.94/95/)
        HBASE-5736 revert TestThriftServer doesn't compile (Revision 1310643)
        HBASE-5736 ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly (Scott Chen) (Revision 1310638)

        Result = FAILURE
        tedyu :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java

        tedyu :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
        Show
        Hudson added a comment - Integrated in HBase-0.94 #95 (See https://builds.apache.org/job/HBase-0.94/95/ ) HBASE-5736 revert TestThriftServer doesn't compile (Revision 1310643) HBASE-5736 ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly (Scott Chen) (Revision 1310638) Result = FAILURE tedyu : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java tedyu : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-security #161 (See https://builds.apache.org/job/HBase-TRUNK-security/161/)
        HBASE-5736 ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly (Scott Chen) (Revision 1310634)

        Result = FAILURE
        tedyu :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-security #161 (See https://builds.apache.org/job/HBase-TRUNK-security/161/ ) HBASE-5736 ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly (Scott Chen) (Revision 1310634) Result = FAILURE tedyu : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94-security #8 (See https://builds.apache.org/job/HBase-0.94-security/8/)
        HBASE-5736 revert TestThriftServer doesn't compile (Revision 1310643)
        HBASE-5736 ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly (Scott Chen) (Revision 1310638)

        Result = SUCCESS
        tedyu :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java

        tedyu :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
        Show
        Hudson added a comment - Integrated in HBase-0.94-security #8 (See https://builds.apache.org/job/HBase-0.94-security/8/ ) HBASE-5736 revert TestThriftServer doesn't compile (Revision 1310643) HBASE-5736 ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly (Scott Chen) (Revision 1310638) Result = SUCCESS tedyu : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java tedyu : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
        Hide
        Ted Yu added a comment -

        Patch for 0.94

        Show
        Ted Yu added a comment - Patch for 0.94
        Hide
        Ted Yu added a comment -

        Integrated patch to 0.94 branch

        Show
        Ted Yu added a comment - Integrated patch to 0.94 branch
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94 #103 (See https://builds.apache.org/job/HBase-0.94/103/)
        HBASE-5736 ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly (Scott Chen) (Revision 1325067)

        Result = SUCCESS
        tedyu :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
        Show
        Hudson added a comment - Integrated in HBase-0.94 #103 (See https://builds.apache.org/job/HBase-0.94/103/ ) HBASE-5736 ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly (Scott Chen) (Revision 1325067) Result = SUCCESS tedyu : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
        Hide
        Lars Hofhansl added a comment -

        Committed, marking as fixed.

        Show
        Lars Hofhansl added a comment - Committed, marking as fixed.
        Hide
        Scott Chen added a comment -

        Thanks for doing the 94 patch

        Show
        Scott Chen added a comment - Thanks for doing the 94 patch
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94-security #9 (See https://builds.apache.org/job/HBase-0.94-security/9/)
        HBASE-5736 ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly (Scott Chen) (Revision 1325067)

        Result = SUCCESS
        tedyu :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
        Show
        Hudson added a comment - Integrated in HBase-0.94-security #9 (See https://builds.apache.org/job/HBase-0.94-security/9/ ) HBASE-5736 ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly (Scott Chen) (Revision 1325067) Result = SUCCESS tedyu : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
        Hide
        Phabricator added a comment -

        sc has closed the revision "HBASE-5736 [jira] ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly".

        REVISION DETAIL
        https://reviews.facebook.net/D2649

        To: tedyu, dhruba, JIRA, sc

        Show
        Phabricator added a comment - sc has closed the revision " HBASE-5736 [jira] ThriftServerRunner.HbaseHandler.mutateRow() does not use ByteBuffer correctly". REVISION DETAIL https://reviews.facebook.net/D2649 To: tedyu, dhruba, JIRA, sc

          People

          • Assignee:
            Scott Chen
            Reporter:
            Scott Chen
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development