Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.95.0
    • Component/s: None
    • Labels:
      None

      Description

      See ScannerCallable. HBASE-7215 comments that portion, but it did not work before, because Results.bytes is no longer used with protobufs.

      1. HBASE-7238-v0.patch
        17 kB
        Sergey Shelukhin

        Activity

        stack made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        stack added a comment -

        Marking closed.

        Show
        stack added a comment - Marking closed.
        stack made changes -
        Fix Version/s 0.98.0 [ 12323143 ]
        stack made changes -
        Fix Version/s 0.98.0 [ 12323143 ]
        stack made changes -
        Fix Version/s 0.95.0 [ 12324094 ]
        Fix Version/s 0.96.0 [ 12320040 ]
        Sergey Shelukhin made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Sergey Shelukhin added a comment -

        Committed.

        Show
        Sergey Shelukhin added a comment - Committed.
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #3761 (See https://builds.apache.org/job/HBase-TRUNK/3761/)
        HBASE-7238 Size based scan metric broken by protobufs (Sergey) (Revision 1434558)

        Result = FAILURE
        tedyu :
        Files :

        • /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java
        • /hbase/trunk/hbase-protocol/src/main/protobuf/Client.proto
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #3761 (See https://builds.apache.org/job/HBase-TRUNK/3761/ ) HBASE-7238 Size based scan metric broken by protobufs (Sergey) (Revision 1434558) Result = FAILURE tedyu : Files : /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java /hbase/trunk/hbase-protocol/src/main/protobuf/Client.proto /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #352 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/352/)
        HBASE-7238 Size based scan metric broken by protobufs (Sergey) (Revision 1434558)

        Result = FAILURE
        tedyu :
        Files :

        • /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java
        • /hbase/trunk/hbase-protocol/src/main/protobuf/Client.proto
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #352 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/352/ ) HBASE-7238 Size based scan metric broken by protobufs (Sergey) (Revision 1434558) Result = FAILURE tedyu : Files : /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java /hbase/trunk/hbase-protocol/src/main/protobuf/Client.proto /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        Hide
        Ted Yu added a comment -

        Integrated to trunk.

        Thanks for the patch, Sergey.

        Thanks for the review, Stack.

        Show
        Ted Yu added a comment - Integrated to trunk. Thanks for the patch, Sergey. Thanks for the review, Stack.
        Show
        stack added a comment - +1 on patch after reading this http://stackoverflow.com/questions/1903490/performance-penalty-of-getserializedsize-in-protocol-buffers
        Hide
        Ted Yu added a comment -

        +1 on patch.

        Show
        Ted Yu added a comment - +1 on patch.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12565245/HBASE-7238-v0.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 lineLengths. The patch introduces lines longer than 100

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.TestLocalHBaseCluster

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4067//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4067//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12565245/HBASE-7238-v0.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces lines longer than 100 -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.TestLocalHBaseCluster Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4067//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4067//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4067//console This message is automatically generated.
        Sergey Shelukhin made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Sergey Shelukhin made changes -
        Attachment HBASE-7238-v0.patch [ 12565245 ]
        Hide
        Sergey Shelukhin added a comment -

        The patch that does what is described below. Same can be done on client, but will incur overhead

        Show
        Sergey Shelukhin added a comment - The patch that does what is described below. Same can be done on client, but will incur overhead
        Hide
        Sergey Shelukhin added a comment -

        Assigning to myself for now, if this is an issue that someone already knowns how to do and just filed for tracking, please feel free to take over

        Show
        Sergey Shelukhin added a comment - Assigning to myself for now, if this is an issue that someone already knowns how to do and just filed for tracking, please feel free to take over
        Sergey Shelukhin made changes -
        Assignee Sergey Shelukhin [ sershe ]
        Hide
        Sergey Shelukhin added a comment -

        I poked around protobuf code. Do we want the wire size of Result-s, in particular?
        Getting that out of the protobuf doesn't look possible via normal interfaces. We can call getSerializedSize on each Result but that may be expensive. We can also call getSerializedSize on them on server side, before adding them to response, and store that inside response too (one long per response); getSerializedSize is memoized on first call, it's called recursively on nested structures, and called during message serialization in RPC (and during write by protobuf), this call is made anyway and thus should not impact perf.

        Show
        Sergey Shelukhin added a comment - I poked around protobuf code. Do we want the wire size of Result-s, in particular? Getting that out of the protobuf doesn't look possible via normal interfaces. We can call getSerializedSize on each Result but that may be expensive. We can also call getSerializedSize on them on server side, before adding them to response, and store that inside response too (one long per response); getSerializedSize is memoized on first call, it's called recursively on nested structures, and called during message serialization in RPC (and during write by protobuf), this call is made anyway and thus should not impact perf.
        stack made changes -
        Field Original Value New Value
        Priority Major [ 3 ] Critical [ 2 ]
        Lars Hofhansl created issue -

          People

          • Assignee:
            Sergey Shelukhin
            Reporter:
            Lars Hofhansl
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development