HBase
  1. HBase
  2. HBASE-2214

Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.95.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Adds Scan#getMaxResultSize
    • Tags:
      0.96notable

      Description

      The notion that you set size rather than row count specifying how many rows a scanner should return in each cycle was raised over in HBASE-1996. Its a good one making hbase "regular" though the data under it may vary. HBASE-1996 was committed but the patch was constrained by the fact that it needed to not change RPC interface. This issue is about doing HBASE-1996 for 0.21 in a clean, unconstrained way.

      1. HBASE-2214-v7.txt
        84 kB
        Ferdy Galema
      2. HBASE-2214-v6.txt
        83 kB
        Ferdy Galema
      3. HBASE-2214-v5.txt
        84 kB
        Ferdy Galema
      4. HBASE-2214-v4.txt
        84 kB
        Ferdy Galema
      5. HBASE-2214-0.94-v3.txt
        8 kB
        Ferdy Galema
      6. HBASE-2214-0.94-v2.txt
        8 kB
        Ferdy Galema
      7. HBASE-2214-0.94.txt
        7 kB
        Ferdy Galema
      8. HBASE-2214_with_broken_TestShell.txt
        61 kB
        Daniel Ploeg

        Issue Links

          Activity

          Hide
          Karthik K added a comment -

          Makes sense conceptually . Would be nice to have a final version of the patch logging the number of rows on the server side when scanning by size , so as to estimate the density / sparsity of the data across columns when we scan by size.

          Show
          Karthik K added a comment - Makes sense conceptually . Would be nice to have a final version of the patch logging the number of rows on the server side when scanning by size , so as to estimate the density / sparsity of the data across columns when we scan by size.
          Hide
          stack added a comment -

          Its HBASE-1996, not 1966.

          Show
          stack added a comment - Its HBASE-1996 , not 1966.
          Hide
          Daniel Ploeg added a comment -

          HBASE-2214_with_broken_TestShell.txt is an early cut. Experiencing problems with this test stopping the process that's running it (eg hudson or ide).

          Show
          Daniel Ploeg added a comment - HBASE-2214 _with_broken_TestShell.txt is an early cut. Experiencing problems with this test stopping the process that's running it (eg hudson or ide).
          Hide
          stack added a comment -

          I tried it locally. It works for me. Is it running out of mem? Its not because you lack MAVEN_OPTS="-Xmx512m" or something?

          Show
          stack added a comment - I tried it locally. It works for me. Is it running out of mem? Its not because you lack MAVEN_OPTS="-Xmx512m" or something?
          Hide
          stack added a comment -

          Moving this 'feature' out.

          Show
          stack added a comment - Moving this 'feature' out.
          Hide
          Ted Yu added a comment -

          @Daniel:
          Are you working on this issue ?

          Show
          Ted Yu added a comment - @Daniel: Are you working on this issue ?
          Hide
          Daniel Ploeg added a comment -

          No - Not working on this issue at present - haven't touched the HBase codebase in quite some time, so the patch here may no longer be relevant. Happy for someone else to take it on, or if it's not relevant anymore to close it out.

          Show
          Daniel Ploeg added a comment - No - Not working on this issue at present - haven't touched the HBase codebase in quite some time, so the patch here may no longer be relevant. Happy for someone else to take it on, or if it's not relevant anymore to close it out.
          Hide
          Ferdy Galema added a comment -

          As mentioned in HBASE-5607, I will pick up this issue.

          First of all:
          I looked at the patch and I noticed that is huge. A lot of unnecessary changes are in it, whitespace/reformatting and such. It is very difficult to work with this when applying it with regular svn patch tools. Therefore, if you don't mind, I want to start over from scratch. (Of course it has been a while so a lot of changes won't apply anyway).

          Secondly, does this feature replaces the old number of rows caching? I like to propose that it is additional. So: A user specifying 100 rows and 10MB for a Scan will get chunks that are either capped at 100 rows, or 10MB, whichever limit comes first. Do you agree?

          Show
          Ferdy Galema added a comment - As mentioned in HBASE-5607 , I will pick up this issue. First of all: I looked at the patch and I noticed that is huge. A lot of unnecessary changes are in it, whitespace/reformatting and such. It is very difficult to work with this when applying it with regular svn patch tools. Therefore, if you don't mind, I want to start over from scratch. (Of course it has been a while so a lot of changes won't apply anyway). Secondly, does this feature replaces the old number of rows caching? I like to propose that it is additional. So: A user specifying 100 rows and 10MB for a Scan will get chunks that are either capped at 100 rows, or 10MB, whichever limit comes first. Do you agree?
          Hide
          Ted Yu added a comment -

          Feel free to start from scratch.

          For #2, size to return is optional setting. In case both size to return and number of rows are specified, your description makes sense.

          Show
          Ted Yu added a comment - Feel free to start from scratch. For #2, size to return is optional setting. In case both size to return and number of rows are specified, your description makes sense.
          Hide
          Ferdy Galema added a comment -

          This is my first attempt. It is not finished, I was just curious if this is the right way to go. Note that it has no tests, because I was not sure where to include those. This patch piggybacks on the existing functionality introduced in HBASE-1996. (It seems that this also has no tests.)

          I tried to add a test to TestHRegion and TestScanner, but it seems the existing tests bypass the HRegionServer "Result[] next(final long scannerId, int nbRows)" method.

          Show
          Ferdy Galema added a comment - This is my first attempt. It is not finished, I was just curious if this is the right way to go. Note that it has no tests, because I was not sure where to include those. This patch piggybacks on the existing functionality introduced in HBASE-1996 . (It seems that this also has no tests.) I tried to add a test to TestHRegion and TestScanner, but it seems the existing tests bypass the HRegionServer "Result[] next(final long scannerId, int nbRows)" method.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12522673/HBASE-2214-0.94.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1525//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1525//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1525//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522673/HBASE-2214-0.94.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1525//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1525//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1525//console This message is automatically generated.
          Hide
          Ted Yu added a comment -
          +   * @return the maximum buffer size in bytes. Also see 
          +   * {@link #setMaxBufferSize(int)}
          

          'Also see' -> 'See also'. The parameter type for setMaxBufferSize() should be long.
          Since line length is 100 now, you can combine the two lines together.

          On review board, it is clear to see the white spaces in the patch.

          Trunk build is currently broken, please submit patch for Hadoop QA when trunk build passes.

          Show
          Ted Yu added a comment - + * @ return the maximum buffer size in bytes. Also see + * {@link #setMaxBufferSize( int )} 'Also see' -> 'See also'. The parameter type for setMaxBufferSize() should be long. Since line length is 100 now, you can combine the two lines together. On review board, it is clear to see the white spaces in the patch. Trunk build is currently broken, please submit patch for Hadoop QA when trunk build passes.
          Hide
          Ferdy Galema added a comment -

          Ok thanks for your comments. I just submitted new patch to review board. https://reviews.apache.org/r/4726/

          So what's protocol here, can I add 'hbase' to reviewers group or just add individual reviewers?

          Show
          Ferdy Galema added a comment - Ok thanks for your comments. I just submitted new patch to review board. https://reviews.apache.org/r/4726/ So what's protocol here, can I add 'hbase' to reviewers group or just add individual reviewers?
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4726/
          -----------------------------------------------------------

          (Updated 2012-04-15 13:38:31.680141)

          Review request for hbase and Ted Yu.

          Summary
          -------

          HBASE-2214 per scan max buffersize.

          This addresses bug HBASE-2214.
          https://issues.apache.org/jira/browse/HBASE-2214

          Diffs


          /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1326084
          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1326084
          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1326084
          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1326084
          /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1326084

          Diff: https://reviews.apache.org/r/4726/diff

          Testing
          -------

          Thanks,

          ferdy

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/ ----------------------------------------------------------- (Updated 2012-04-15 13:38:31.680141) Review request for hbase and Ted Yu. Summary ------- HBASE-2214 per scan max buffersize. This addresses bug HBASE-2214 . https://issues.apache.org/jira/browse/HBASE-2214 Diffs /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1326084 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1326084 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1326084 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1326084 /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1326084 Diff: https://reviews.apache.org/r/4726/diff Testing ------- Thanks, ferdy
          Hide
          Ferdy Galema added a comment -

          Secondly, I'd like the uploaded diff to be compared to the 0.94 branch. How can I specify this? It seems it is always compared to trunk?

          Show
          Ferdy Galema added a comment - Secondly, I'd like the uploaded diff to be compared to the 0.94 branch. How can I specify this? It seems it is always compared to trunk?
          Hide
          Ted Yu added a comment -

          'hbase' group should be included in each review request.
          Normally patch for trunk is generated first and gets reviewed on review board. Hadoop QA checks out patch against trunk.

          Show
          Ted Yu added a comment - 'hbase' group should be included in each review request. Normally patch for trunk is generated first and gets reviewed on review board. Hadoop QA checks out patch against trunk.
          Hide
          Ted Yu added a comment -

          I went over https://reviews.apache.org/r/4726 which looks clean.

          If adding unit test is difficult, can you perform verification on real clsuter and let us know the results ?

          Show
          Ted Yu added a comment - I went over https://reviews.apache.org/r/4726 which looks clean. If adding unit test is difficult, can you perform verification on real clsuter and let us know the results ?
          Hide
          Ferdy Galema added a comment -

          Sure I will do that.

          Some patch segments do not apply to trunk so I will first create a patch for trunk.

          Show
          Ferdy Galema added a comment - Sure I will do that. Some patch segments do not apply to trunk so I will first create a patch for trunk.
          Hide
          Ferdy Galema added a comment -

          (Nevermind the latest comment. It seems that with a fuzz factor it does apply.)

          Show
          Ferdy Galema added a comment - (Nevermind the latest comment. It seems that with a fuzz factor it does apply.)
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4726/#review6968
          -----------------------------------------------------------

          I think this an important addition. Any chance of a test Ferdy to demonstrate it works. Its kinda crazy you only have to do this much to make it work.

          /src/main/java/org/apache/hadoop/hbase/client/Scan.java
          <https://reviews.apache.org/r/4726/#comment15518>

          Good

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
          <https://reviews.apache.org/r/4726/#comment15519>

          Needs to be public?

          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java
          <https://reviews.apache.org/r/4726/#comment15520>

          This is not used?

          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java
          <https://reviews.apache.org/r/4726/#comment15521>

          Is this a good name for this attribute? Its the result size per next? getMaxResultSize? Or something like that?

          • Michael

          On 2012-04-15 13:38:31, ferdy wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/4726/

          -----------------------------------------------------------

          (Updated 2012-04-15 13:38:31)

          Review request for hbase and Ted Yu.

          Summary

          -------

          HBASE-2214 per scan max buffersize.

          This addresses bug HBASE-2214.

          https://issues.apache.org/jira/browse/HBASE-2214

          Diffs

          -----

          /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1326084

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1326084

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1326084

          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1326084

          /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1326084

          Diff: https://reviews.apache.org/r/4726/diff

          Testing

          -------

          Thanks,

          ferdy

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/#review6968 ----------------------------------------------------------- I think this an important addition. Any chance of a test Ferdy to demonstrate it works. Its kinda crazy you only have to do this much to make it work. /src/main/java/org/apache/hadoop/hbase/client/Scan.java < https://reviews.apache.org/r/4726/#comment15518 > Good /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java < https://reviews.apache.org/r/4726/#comment15519 > Needs to be public? /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java < https://reviews.apache.org/r/4726/#comment15520 > This is not used? /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java < https://reviews.apache.org/r/4726/#comment15521 > Is this a good name for this attribute? Its the result size per next? getMaxResultSize? Or something like that? Michael On 2012-04-15 13:38:31, ferdy wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/ ----------------------------------------------------------- (Updated 2012-04-15 13:38:31) Review request for hbase and Ted Yu. Summary ------- HBASE-2214 per scan max buffersize. This addresses bug HBASE-2214 . https://issues.apache.org/jira/browse/HBASE-2214 Diffs ----- /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1326084 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1326084 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1326084 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1326084 /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1326084 Diff: https://reviews.apache.org/r/4726/diff Testing ------- Thanks, ferdy
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2012-04-17 05:46:43, Michael Stack wrote:

          > I think this an important addition. Any chance of a test Ferdy to demonstrate it works. Its kinda crazy you only have to do this much to make it work.

          It was indeed to good to be true. It does not yet work. There are still some other classes that need modification. (In order to make it work by piggybacking on the existing maxScannerResultSize mechanism).

          Will get back at this with new version of patch.

          Thanks.

          On 2012-04-17 05:46:43, Michael Stack wrote:

          > /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java, line 26

          > <https://reviews.apache.org/r/4726/diff/2/?file=101902#file101902line26>

          >

          > This is not used?

          It is, namely in the javadoc comment. Removing it makes the link unclickable.

          On 2012-04-17 05:46:43, Michael Stack wrote:

          > /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java, line 3376

          > <https://reviews.apache.org/r/4726/diff/2/?file=101900#file101900line3376>

          >

          > Needs to be public?

          I cannot reduce the visibilty because it needs to in the interface. (Yeah it's a bit of a hack I guess)

          On 2012-04-17 05:46:43, Michael Stack wrote:

          > /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java, line 59

          > <https://reviews.apache.org/r/4726/diff/2/?file=101902#file101902line59>

          >

          > Is this a good name for this attribute? Its the result size per next? getMaxResultSize? Or something like that?

          maxResultSize seems better indeed. I will rename this next patch.

          • ferdy

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4726/#review6968
          -----------------------------------------------------------

          On 2012-04-15 13:38:31, ferdy wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/4726/

          -----------------------------------------------------------

          (Updated 2012-04-15 13:38:31)

          Review request for hbase and Ted Yu.

          Summary

          -------

          HBASE-2214 per scan max buffersize.

          This addresses bug HBASE-2214.

          https://issues.apache.org/jira/browse/HBASE-2214

          Diffs

          -----

          /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1326084

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1326084

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1326084

          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1326084

          /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1326084

          Diff: https://reviews.apache.org/r/4726/diff

          Testing

          -------

          Thanks,

          ferdy

          Show
          jiraposter@reviews.apache.org added a comment - On 2012-04-17 05:46:43, Michael Stack wrote: > I think this an important addition. Any chance of a test Ferdy to demonstrate it works. Its kinda crazy you only have to do this much to make it work. It was indeed to good to be true. It does not yet work. There are still some other classes that need modification. (In order to make it work by piggybacking on the existing maxScannerResultSize mechanism). Will get back at this with new version of patch. Thanks. On 2012-04-17 05:46:43, Michael Stack wrote: > /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java, line 26 > < https://reviews.apache.org/r/4726/diff/2/?file=101902#file101902line26 > > > This is not used? It is, namely in the javadoc comment. Removing it makes the link unclickable. On 2012-04-17 05:46:43, Michael Stack wrote: > /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java, line 3376 > < https://reviews.apache.org/r/4726/diff/2/?file=101900#file101900line3376 > > > Needs to be public? I cannot reduce the visibilty because it needs to in the interface. (Yeah it's a bit of a hack I guess) On 2012-04-17 05:46:43, Michael Stack wrote: > /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java, line 59 > < https://reviews.apache.org/r/4726/diff/2/?file=101902#file101902line59 > > > Is this a good name for this attribute? Its the result size per next? getMaxResultSize? Or something like that? maxResultSize seems better indeed. I will rename this next patch. ferdy ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/#review6968 ----------------------------------------------------------- On 2012-04-15 13:38:31, ferdy wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/ ----------------------------------------------------------- (Updated 2012-04-15 13:38:31) Review request for hbase and Ted Yu. Summary ------- HBASE-2214 per scan max buffersize. This addresses bug HBASE-2214 . https://issues.apache.org/jira/browse/HBASE-2214 Diffs ----- /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1326084 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1326084 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1326084 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1326084 /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1326084 Diff: https://reviews.apache.org/r/4726/diff Testing ------- Thanks, ferdy
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4726/
          -----------------------------------------------------------

          (Updated 2012-04-25 20:25:06.141778)

          Review request for hbase and Ted Yu.

          Changes
          -------

          Upload v4 of patch.

          Note that patch includes updated generated protobuffers sources.

          Summary
          -------

          HBASE-2214 per scan max buffersize.

          This addresses bug HBASE-2214.
          https://issues.apache.org/jira/browse/HBASE-2214

          Diffs (updated)


          /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330504
          /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330504
          /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330504
          /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330504
          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330504
          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330504
          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330504
          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330504
          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330504
          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1330504
          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1330504
          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1330504
          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1330504
          /src/main/protobuf/Client.proto 1330504
          /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1330504

          Diff: https://reviews.apache.org/r/4726/diff

          Testing (updated)
          -------

          It works when running this test:

          new HBaseTestingUtility(conf).startMiniCluster();

          HBaseAdmin admin = new HBaseAdmin(conf);
          if (!admin.tableExists("test"))

          { HTableDescriptor tableDesc = new HTableDescriptor("test"); tableDesc.addFamily(new HColumnDescriptor("fam")); admin.createTable(tableDesc); }

          HTable table = new HTable(conf, "test");
          Put put;

          put = new Put(Bytes.toBytes("row1"));
          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1"));
          table.put(put);

          put = new Put(Bytes.toBytes("row2"));
          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2"));
          table.put(put);

          put = new Put(Bytes.toBytes("row3"));
          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3"));
          table.put(put);

          table.flushCommits();

          { System.out.println("returns all rows at once because of the caching"); Scan scan = new Scan(); scan.setCaching(100); ResultScanner scanner = table.getScanner(scan); scanner.next(100); } { System.out.println("returns one row at a time because of the maxResultSize"); Scan scan = new Scan(); scan.setCaching(100); scan.setMaxResultSize(1); ResultScanner scanner = table.getScanner(scan); scanner.next(100); }

          See output:

          returns all rows at once because of the caching
          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''
          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''
          2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3
          2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at

          {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,}

          returns one row at a time because of the maxResultSize
          2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''
          2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''
          2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1
          2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1
          2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1

          Thanks,

          ferdy

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/ ----------------------------------------------------------- (Updated 2012-04-25 20:25:06.141778) Review request for hbase and Ted Yu. Changes ------- Upload v4 of patch. Note that patch includes updated generated protobuffers sources. Summary ------- HBASE-2214 per scan max buffersize. This addresses bug HBASE-2214 . https://issues.apache.org/jira/browse/HBASE-2214 Diffs (updated) /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330504 /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330504 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1330504 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1330504 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1330504 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1330504 /src/main/protobuf/Client.proto 1330504 /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1330504 Diff: https://reviews.apache.org/r/4726/diff Testing (updated) ------- It works when running this test: new HBaseTestingUtility(conf).startMiniCluster(); HBaseAdmin admin = new HBaseAdmin(conf); if (!admin.tableExists("test")) { HTableDescriptor tableDesc = new HTableDescriptor("test"); tableDesc.addFamily(new HColumnDescriptor("fam")); admin.createTable(tableDesc); } HTable table = new HTable(conf, "test"); Put put; put = new Put(Bytes.toBytes("row1")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1")); table.put(put); put = new Put(Bytes.toBytes("row2")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2")); table.put(put); put = new Put(Bytes.toBytes("row3")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3")); table.put(put); table.flushCommits(); { System.out.println("returns all rows at once because of the caching"); Scan scan = new Scan(); scan.setCaching(100); ResultScanner scanner = table.getScanner(scan); scanner.next(100); } { System.out.println("returns one row at a time because of the maxResultSize"); Scan scan = new Scan(); scan.setCaching(100); scan.setMaxResultSize(1); ResultScanner scanner = table.getScanner(scan); scanner.next(100); } See output: returns all rows at once because of the caching 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3 2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,} returns one row at a time because of the maxResultSize 2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1 Thanks, ferdy
          Hide
          Ferdy Galema added a comment -

          New version of patch. This works when manually running a test. (See review board for testing method)

          Show
          Ferdy Galema added a comment - New version of patch. This works when manually running a test. (See review board for testing method)
          Hide
          Ferdy Galema added a comment -

          (patch was for trunk)

          Show
          Ferdy Galema added a comment - (patch was for trunk)
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4726/#review7242
          -----------------------------------------------------------

          lgtm.

          /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
          <https://reviews.apache.org/r/4726/#comment15974>

          This should be removed.

          • Ted

          On 2012-04-25 20:25:06, ferdy wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/4726/

          -----------------------------------------------------------

          (Updated 2012-04-25 20:25:06)

          Review request for hbase and Ted Yu.

          Summary

          -------

          HBASE-2214 per scan max buffersize.

          This addresses bug HBASE-2214.

          https://issues.apache.org/jira/browse/HBASE-2214

          Diffs

          -----

          /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330504

          /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330504

          /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330504

          /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330504

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330504

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330504

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330504

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330504

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330504

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1330504

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1330504

          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1330504

          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1330504

          /src/main/protobuf/Client.proto 1330504

          /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1330504

          Diff: https://reviews.apache.org/r/4726/diff

          Testing

          -------

          It works when running this test:

          new HBaseTestingUtility(conf).startMiniCluster();

          HBaseAdmin admin = new HBaseAdmin(conf);

          if (!admin.tableExists("test")) { bq. HTableDescriptor tableDesc = new HTableDescriptor("test"); bq. tableDesc.addFamily(new HColumnDescriptor("fam")); bq. admin.createTable(tableDesc); bq. }

          HTable table = new HTable(conf, "test");

          Put put;

          put = new Put(Bytes.toBytes("row1"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1"));

          table.put(put);

          put = new Put(Bytes.toBytes("row2"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2"));

          table.put(put);

          put = new Put(Bytes.toBytes("row3"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3"));

          table.put(put);

          table.flushCommits();

          { bq. System.out.println("returns all rows at once because of the caching"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. }

          { bq. System.out.println("returns one row at a time because of the maxResultSize"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. scan.setMaxResultSize(1); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. }

          See output:

          returns all rows at once because of the caching

          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''

          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''

          2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3

          2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,}

          returns one row at a time because of the maxResultSize

          2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''

          2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''

          2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1

          2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1

          2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1

          Thanks,

          ferdy

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/#review7242 ----------------------------------------------------------- lgtm. /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java < https://reviews.apache.org/r/4726/#comment15974 > This should be removed. Ted On 2012-04-25 20:25:06, ferdy wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/ ----------------------------------------------------------- (Updated 2012-04-25 20:25:06) Review request for hbase and Ted Yu. Summary ------- HBASE-2214 per scan max buffersize. This addresses bug HBASE-2214 . https://issues.apache.org/jira/browse/HBASE-2214 Diffs ----- /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330504 /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330504 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1330504 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1330504 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1330504 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1330504 /src/main/protobuf/Client.proto 1330504 /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1330504 Diff: https://reviews.apache.org/r/4726/diff Testing ------- It works when running this test: new HBaseTestingUtility(conf).startMiniCluster(); HBaseAdmin admin = new HBaseAdmin(conf); if (!admin.tableExists("test")) { bq. HTableDescriptor tableDesc = new HTableDescriptor("test"); bq. tableDesc.addFamily(new HColumnDescriptor("fam")); bq. admin.createTable(tableDesc); bq. } HTable table = new HTable(conf, "test"); Put put; put = new Put(Bytes.toBytes("row1")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1")); table.put(put); put = new Put(Bytes.toBytes("row2")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2")); table.put(put); put = new Put(Bytes.toBytes("row3")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3")); table.put(put); table.flushCommits(); { bq. System.out.println("returns all rows at once because of the caching"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. } { bq. System.out.println("returns one row at a time because of the maxResultSize"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. scan.setMaxResultSize(1); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. } See output: returns all rows at once because of the caching 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3 2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,} returns one row at a time because of the maxResultSize 2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1 Thanks, ferdy
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12524343/HBASE-2214-v4.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1652//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1652//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1652//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12524343/HBASE-2214-v4.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1652//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1652//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1652//console This message is automatically generated.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4726/#review7249
          -----------------------------------------------------------

          /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
          <https://reviews.apache.org/r/4726/#comment16020>

          Can we set it only if maxResultSize is > 0?

          /src/main/protobuf/Client.proto
          <https://reviews.apache.org/r/4726/#comment16019>

          Can we use uint64, without a default? So if it is not specified, we take it as -1.

          • Jimmy

          On 2012-04-25 20:25:06, ferdy wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/4726/

          -----------------------------------------------------------

          (Updated 2012-04-25 20:25:06)

          Review request for hbase and Ted Yu.

          Summary

          -------

          HBASE-2214 per scan max buffersize.

          This addresses bug HBASE-2214.

          https://issues.apache.org/jira/browse/HBASE-2214

          Diffs

          -----

          /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330504

          /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330504

          /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330504

          /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330504

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330504

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330504

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330504

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330504

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330504

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1330504

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1330504

          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1330504

          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1330504

          /src/main/protobuf/Client.proto 1330504

          /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1330504

          Diff: https://reviews.apache.org/r/4726/diff

          Testing

          -------

          It works when running this test:

          new HBaseTestingUtility(conf).startMiniCluster();

          HBaseAdmin admin = new HBaseAdmin(conf);

          if (!admin.tableExists("test")) { bq. HTableDescriptor tableDesc = new HTableDescriptor("test"); bq. tableDesc.addFamily(new HColumnDescriptor("fam")); bq. admin.createTable(tableDesc); bq. }

          HTable table = new HTable(conf, "test");

          Put put;

          put = new Put(Bytes.toBytes("row1"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1"));

          table.put(put);

          put = new Put(Bytes.toBytes("row2"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2"));

          table.put(put);

          put = new Put(Bytes.toBytes("row3"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3"));

          table.put(put);

          table.flushCommits();

          { bq. System.out.println("returns all rows at once because of the caching"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. }

          { bq. System.out.println("returns one row at a time because of the maxResultSize"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. scan.setMaxResultSize(1); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. }

          See output:

          returns all rows at once because of the caching

          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''

          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''

          2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3

          2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,}

          returns one row at a time because of the maxResultSize

          2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''

          2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''

          2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1

          2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1

          2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1

          Thanks,

          ferdy

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/#review7249 ----------------------------------------------------------- /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java < https://reviews.apache.org/r/4726/#comment16020 > Can we set it only if maxResultSize is > 0? /src/main/protobuf/Client.proto < https://reviews.apache.org/r/4726/#comment16019 > Can we use uint64, without a default? So if it is not specified, we take it as -1. Jimmy On 2012-04-25 20:25:06, ferdy wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/ ----------------------------------------------------------- (Updated 2012-04-25 20:25:06) Review request for hbase and Ted Yu. Summary ------- HBASE-2214 per scan max buffersize. This addresses bug HBASE-2214 . https://issues.apache.org/jira/browse/HBASE-2214 Diffs ----- /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330504 /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330504 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330504 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1330504 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1330504 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1330504 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1330504 /src/main/protobuf/Client.proto 1330504 /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1330504 Diff: https://reviews.apache.org/r/4726/diff Testing ------- It works when running this test: new HBaseTestingUtility(conf).startMiniCluster(); HBaseAdmin admin = new HBaseAdmin(conf); if (!admin.tableExists("test")) { bq. HTableDescriptor tableDesc = new HTableDescriptor("test"); bq. tableDesc.addFamily(new HColumnDescriptor("fam")); bq. admin.createTable(tableDesc); bq. } HTable table = new HTable(conf, "test"); Put put; put = new Put(Bytes.toBytes("row1")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1")); table.put(put); put = new Put(Bytes.toBytes("row2")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2")); table.put(put); put = new Put(Bytes.toBytes("row3")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3")); table.put(put); table.flushCommits(); { bq. System.out.println("returns all rows at once because of the caching"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. } { bq. System.out.println("returns one row at a time because of the maxResultSize"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. scan.setMaxResultSize(1); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. } See output: returns all rows at once because of the caching 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3 2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,} returns one row at a time because of the maxResultSize 2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1 Thanks, ferdy
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4726/
          -----------------------------------------------------------

          (Updated 2012-04-26 08:18:40.576784)

          Review request for hbase and Ted Yu.

          Changes
          -------

          Uploaded v5 of patch. The test still works.

          Summary
          -------

          HBASE-2214 per scan max buffersize.

          This addresses bug HBASE-2214.
          https://issues.apache.org/jira/browse/HBASE-2214

          Diffs (updated)


          /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330680
          /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330680
          /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330680
          /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330680
          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330680
          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330680
          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330680
          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330680
          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330680
          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1330680
          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1330680
          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1330680
          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1330680
          /src/main/protobuf/Client.proto 1330680
          /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1330680

          Diff: https://reviews.apache.org/r/4726/diff

          Testing
          -------

          It works when running this test:

          new HBaseTestingUtility(conf).startMiniCluster();

          HBaseAdmin admin = new HBaseAdmin(conf);
          if (!admin.tableExists("test"))

          { HTableDescriptor tableDesc = new HTableDescriptor("test"); tableDesc.addFamily(new HColumnDescriptor("fam")); admin.createTable(tableDesc); }

          HTable table = new HTable(conf, "test");
          Put put;

          put = new Put(Bytes.toBytes("row1"));
          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1"));
          table.put(put);

          put = new Put(Bytes.toBytes("row2"));
          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2"));
          table.put(put);

          put = new Put(Bytes.toBytes("row3"));
          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3"));
          table.put(put);

          table.flushCommits();

          { System.out.println("returns all rows at once because of the caching"); Scan scan = new Scan(); scan.setCaching(100); ResultScanner scanner = table.getScanner(scan); scanner.next(100); } { System.out.println("returns one row at a time because of the maxResultSize"); Scan scan = new Scan(); scan.setCaching(100); scan.setMaxResultSize(1); ResultScanner scanner = table.getScanner(scan); scanner.next(100); }

          See output:

          returns all rows at once because of the caching
          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''
          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''
          2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3
          2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at

          {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,}

          returns one row at a time because of the maxResultSize
          2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''
          2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''
          2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1
          2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1
          2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1

          Thanks,

          ferdy

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/ ----------------------------------------------------------- (Updated 2012-04-26 08:18:40.576784) Review request for hbase and Ted Yu. Changes ------- Uploaded v5 of patch. The test still works. Summary ------- HBASE-2214 per scan max buffersize. This addresses bug HBASE-2214 . https://issues.apache.org/jira/browse/HBASE-2214 Diffs (updated) /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330680 /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1330680 /src/main/protobuf/Client.proto 1330680 /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1330680 Diff: https://reviews.apache.org/r/4726/diff Testing ------- It works when running this test: new HBaseTestingUtility(conf).startMiniCluster(); HBaseAdmin admin = new HBaseAdmin(conf); if (!admin.tableExists("test")) { HTableDescriptor tableDesc = new HTableDescriptor("test"); tableDesc.addFamily(new HColumnDescriptor("fam")); admin.createTable(tableDesc); } HTable table = new HTable(conf, "test"); Put put; put = new Put(Bytes.toBytes("row1")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1")); table.put(put); put = new Put(Bytes.toBytes("row2")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2")); table.put(put); put = new Put(Bytes.toBytes("row3")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3")); table.put(put); table.flushCommits(); { System.out.println("returns all rows at once because of the caching"); Scan scan = new Scan(); scan.setCaching(100); ResultScanner scanner = table.getScanner(scan); scanner.next(100); } { System.out.println("returns one row at a time because of the maxResultSize"); Scan scan = new Scan(); scan.setCaching(100); scan.setMaxResultSize(1); ResultScanner scanner = table.getScanner(scan); scanner.next(100); } See output: returns all rows at once because of the caching 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3 2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,} returns one row at a time because of the maxResultSize 2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1 Thanks, ferdy
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2012-04-25 21:02:15, Ted Yu wrote:

          > /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java, line 323

          > <https://reviews.apache.org/r/4726/diff/3/?file=104298#file104298line323>

          >

          > This should be removed.

          Ok done.

          • ferdy

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4726/#review7242
          -----------------------------------------------------------

          On 2012-04-26 08:18:40, ferdy wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/4726/

          -----------------------------------------------------------

          (Updated 2012-04-26 08:18:40)

          Review request for hbase and Ted Yu.

          Summary

          -------

          HBASE-2214 per scan max buffersize.

          This addresses bug HBASE-2214.

          https://issues.apache.org/jira/browse/HBASE-2214

          Diffs

          -----

          /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330680

          /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1330680

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1330680

          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1330680

          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1330680

          /src/main/protobuf/Client.proto 1330680

          /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1330680

          Diff: https://reviews.apache.org/r/4726/diff

          Testing

          -------

          It works when running this test:

          new HBaseTestingUtility(conf).startMiniCluster();

          HBaseAdmin admin = new HBaseAdmin(conf);

          if (!admin.tableExists("test")) { bq. HTableDescriptor tableDesc = new HTableDescriptor("test"); bq. tableDesc.addFamily(new HColumnDescriptor("fam")); bq. admin.createTable(tableDesc); bq. }

          HTable table = new HTable(conf, "test");

          Put put;

          put = new Put(Bytes.toBytes("row1"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1"));

          table.put(put);

          put = new Put(Bytes.toBytes("row2"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2"));

          table.put(put);

          put = new Put(Bytes.toBytes("row3"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3"));

          table.put(put);

          table.flushCommits();

          { bq. System.out.println("returns all rows at once because of the caching"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. }

          { bq. System.out.println("returns one row at a time because of the maxResultSize"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. scan.setMaxResultSize(1); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. }

          See output:

          returns all rows at once because of the caching

          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''

          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''

          2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3

          2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,}

          returns one row at a time because of the maxResultSize

          2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''

          2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''

          2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1

          2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1

          2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1

          Thanks,

          ferdy

          Show
          jiraposter@reviews.apache.org added a comment - On 2012-04-25 21:02:15, Ted Yu wrote: > /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java, line 323 > < https://reviews.apache.org/r/4726/diff/3/?file=104298#file104298line323 > > > This should be removed. Ok done. ferdy ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/#review7242 ----------------------------------------------------------- On 2012-04-26 08:18:40, ferdy wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/ ----------------------------------------------------------- (Updated 2012-04-26 08:18:40) Review request for hbase and Ted Yu. Summary ------- HBASE-2214 per scan max buffersize. This addresses bug HBASE-2214 . https://issues.apache.org/jira/browse/HBASE-2214 Diffs ----- /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330680 /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1330680 /src/main/protobuf/Client.proto 1330680 /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1330680 Diff: https://reviews.apache.org/r/4726/diff Testing ------- It works when running this test: new HBaseTestingUtility(conf).startMiniCluster(); HBaseAdmin admin = new HBaseAdmin(conf); if (!admin.tableExists("test")) { bq. HTableDescriptor tableDesc = new HTableDescriptor("test"); bq. tableDesc.addFamily(new HColumnDescriptor("fam")); bq. admin.createTable(tableDesc); bq. } HTable table = new HTable(conf, "test"); Put put; put = new Put(Bytes.toBytes("row1")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1")); table.put(put); put = new Put(Bytes.toBytes("row2")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2")); table.put(put); put = new Put(Bytes.toBytes("row3")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3")); table.put(put); table.flushCommits(); { bq. System.out.println("returns all rows at once because of the caching"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. } { bq. System.out.println("returns one row at a time because of the maxResultSize"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. scan.setMaxResultSize(1); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. } See output: returns all rows at once because of the caching 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3 2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,} returns one row at a time because of the maxResultSize 2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1 Thanks, ferdy
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2012-04-26 03:10:35, Jimmy Xiang wrote:

          > /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java, line 385

          > <https://reviews.apache.org/r/4726/diff/3/?file=104301#file104301line385>

          >

          > Can we set it only if maxResultSize is > 0?

          Done. I've changed it everywhere else too so that maxResultSize only works when > 0. (Setting to 0 is bogus anyway, it might cause infinite loops).

          On 2012-04-26 03:10:35, Jimmy Xiang wrote:

          > /src/main/protobuf/Client.proto, line 197

          > <https://reviews.apache.org/r/4726/diff/3/?file=104311#file104311line197>

          >

          > Can we use uint64, without a default? So if it is not specified, we take it as -1.

          Done. (Including regenerating the protobuffers sources). I already noticed that int64 was not used anywhere else in the proto file. Just out of curiosity, are there good reasons to avoid int64?

          • ferdy

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4726/#review7249
          -----------------------------------------------------------

          On 2012-04-26 08:18:40, ferdy wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/4726/

          -----------------------------------------------------------

          (Updated 2012-04-26 08:18:40)

          Review request for hbase and Ted Yu.

          Summary

          -------

          HBASE-2214 per scan max buffersize.

          This addresses bug HBASE-2214.

          https://issues.apache.org/jira/browse/HBASE-2214

          Diffs

          -----

          /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330680

          /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1330680

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1330680

          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1330680

          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1330680

          /src/main/protobuf/Client.proto 1330680

          /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1330680

          Diff: https://reviews.apache.org/r/4726/diff

          Testing

          -------

          It works when running this test:

          new HBaseTestingUtility(conf).startMiniCluster();

          HBaseAdmin admin = new HBaseAdmin(conf);

          if (!admin.tableExists("test")) { bq. HTableDescriptor tableDesc = new HTableDescriptor("test"); bq. tableDesc.addFamily(new HColumnDescriptor("fam")); bq. admin.createTable(tableDesc); bq. }

          HTable table = new HTable(conf, "test");

          Put put;

          put = new Put(Bytes.toBytes("row1"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1"));

          table.put(put);

          put = new Put(Bytes.toBytes("row2"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2"));

          table.put(put);

          put = new Put(Bytes.toBytes("row3"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3"));

          table.put(put);

          table.flushCommits();

          { bq. System.out.println("returns all rows at once because of the caching"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. }

          { bq. System.out.println("returns one row at a time because of the maxResultSize"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. scan.setMaxResultSize(1); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. }

          See output:

          returns all rows at once because of the caching

          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''

          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''

          2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3

          2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,}

          returns one row at a time because of the maxResultSize

          2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''

          2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''

          2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1

          2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1

          2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1

          Thanks,

          ferdy

          Show
          jiraposter@reviews.apache.org added a comment - On 2012-04-26 03:10:35, Jimmy Xiang wrote: > /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java, line 385 > < https://reviews.apache.org/r/4726/diff/3/?file=104301#file104301line385 > > > Can we set it only if maxResultSize is > 0? Done. I've changed it everywhere else too so that maxResultSize only works when > 0. (Setting to 0 is bogus anyway, it might cause infinite loops). On 2012-04-26 03:10:35, Jimmy Xiang wrote: > /src/main/protobuf/Client.proto, line 197 > < https://reviews.apache.org/r/4726/diff/3/?file=104311#file104311line197 > > > Can we use uint64, without a default? So if it is not specified, we take it as -1. Done. (Including regenerating the protobuffers sources). I already noticed that int64 was not used anywhere else in the proto file. Just out of curiosity, are there good reasons to avoid int64? ferdy ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/#review7249 ----------------------------------------------------------- On 2012-04-26 08:18:40, ferdy wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/ ----------------------------------------------------------- (Updated 2012-04-26 08:18:40) Review request for hbase and Ted Yu. Summary ------- HBASE-2214 per scan max buffersize. This addresses bug HBASE-2214 . https://issues.apache.org/jira/browse/HBASE-2214 Diffs ----- /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330680 /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1330680 /src/main/protobuf/Client.proto 1330680 /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1330680 Diff: https://reviews.apache.org/r/4726/diff Testing ------- It works when running this test: new HBaseTestingUtility(conf).startMiniCluster(); HBaseAdmin admin = new HBaseAdmin(conf); if (!admin.tableExists("test")) { bq. HTableDescriptor tableDesc = new HTableDescriptor("test"); bq. tableDesc.addFamily(new HColumnDescriptor("fam")); bq. admin.createTable(tableDesc); bq. } HTable table = new HTable(conf, "test"); Put put; put = new Put(Bytes.toBytes("row1")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1")); table.put(put); put = new Put(Bytes.toBytes("row2")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2")); table.put(put); put = new Put(Bytes.toBytes("row3")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3")); table.put(put); table.flushCommits(); { bq. System.out.println("returns all rows at once because of the caching"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. } { bq. System.out.println("returns one row at a time because of the maxResultSize"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. scan.setMaxResultSize(1); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. } See output: returns all rows at once because of the caching 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3 2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,} returns one row at a time because of the maxResultSize 2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1 Thanks, ferdy
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12524407/HBASE-2214-v5.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.master.TestSplitLogManager

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1654//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1654//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1654//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12524407/HBASE-2214-v5.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestSplitLogManager Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1654//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1654//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1654//console This message is automatically generated.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4726/#review7268
          -----------------------------------------------------------

          Looks good. It seems you are using an old version protoc, which is ok. Great stuff.

          • Jimmy

          On 2012-04-26 08:18:40, ferdy wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/4726/

          -----------------------------------------------------------

          (Updated 2012-04-26 08:18:40)

          Review request for hbase and Ted Yu.

          Summary

          -------

          HBASE-2214 per scan max buffersize.

          This addresses bug HBASE-2214.

          https://issues.apache.org/jira/browse/HBASE-2214

          Diffs

          -----

          /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330680

          /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1330680

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1330680

          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1330680

          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1330680

          /src/main/protobuf/Client.proto 1330680

          /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1330680

          Diff: https://reviews.apache.org/r/4726/diff

          Testing

          -------

          It works when running this test:

          new HBaseTestingUtility(conf).startMiniCluster();

          HBaseAdmin admin = new HBaseAdmin(conf);

          if (!admin.tableExists("test")) { bq. HTableDescriptor tableDesc = new HTableDescriptor("test"); bq. tableDesc.addFamily(new HColumnDescriptor("fam")); bq. admin.createTable(tableDesc); bq. }

          HTable table = new HTable(conf, "test");

          Put put;

          put = new Put(Bytes.toBytes("row1"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1"));

          table.put(put);

          put = new Put(Bytes.toBytes("row2"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2"));

          table.put(put);

          put = new Put(Bytes.toBytes("row3"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3"));

          table.put(put);

          table.flushCommits();

          { bq. System.out.println("returns all rows at once because of the caching"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. }

          { bq. System.out.println("returns one row at a time because of the maxResultSize"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. scan.setMaxResultSize(1); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. }

          See output:

          returns all rows at once because of the caching

          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''

          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''

          2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3

          2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,}

          returns one row at a time because of the maxResultSize

          2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''

          2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''

          2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1

          2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1

          2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1

          Thanks,

          ferdy

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/#review7268 ----------------------------------------------------------- Looks good. It seems you are using an old version protoc, which is ok. Great stuff. Jimmy On 2012-04-26 08:18:40, ferdy wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/ ----------------------------------------------------------- (Updated 2012-04-26 08:18:40) Review request for hbase and Ted Yu. Summary ------- HBASE-2214 per scan max buffersize. This addresses bug HBASE-2214 . https://issues.apache.org/jira/browse/HBASE-2214 Diffs ----- /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330680 /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1330680 /src/main/protobuf/Client.proto 1330680 /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1330680 Diff: https://reviews.apache.org/r/4726/diff Testing ------- It works when running this test: new HBaseTestingUtility(conf).startMiniCluster(); HBaseAdmin admin = new HBaseAdmin(conf); if (!admin.tableExists("test")) { bq. HTableDescriptor tableDesc = new HTableDescriptor("test"); bq. tableDesc.addFamily(new HColumnDescriptor("fam")); bq. admin.createTable(tableDesc); bq. } HTable table = new HTable(conf, "test"); Put put; put = new Put(Bytes.toBytes("row1")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1")); table.put(put); put = new Put(Bytes.toBytes("row2")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2")); table.put(put); put = new Put(Bytes.toBytes("row3")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3")); table.put(put); table.flushCommits(); { bq. System.out.println("returns all rows at once because of the caching"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. } { bq. System.out.println("returns one row at a time because of the maxResultSize"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. scan.setMaxResultSize(1); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. } See output: returns all rows at once because of the caching 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3 2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,} returns one row at a time because of the maxResultSize 2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1 Thanks, ferdy
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4726/#review7383
          -----------------------------------------------------------

          Where are we checking the size of the result made so far? I don't see it in the below. I'd expect it inside in the RegionScanner. Any chance of a test? Otherwise, patch looks great.

          /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
          <https://reviews.apache.org/r/4726/#comment16293>

          Is this going to be annoying? If a high-traffic server, won't this get logged once per request? Perhaps thousands a second?

          /src/main/java/org/apache/hadoop/hbase/client/Scan.java
          <https://reviews.apache.org/r/4726/#comment16294>

          Is this needed? Is this set on Scan creation? When would it change after Scan construction? Or, are we using builder pattern here and its set after construction but before use?

          /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
          <https://reviews.apache.org/r/4726/#comment16295>

          oh, I see how its used now. ignore above comment.

          • Michael

          On 2012-04-26 08:18:40, ferdy wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/4726/

          -----------------------------------------------------------

          (Updated 2012-04-26 08:18:40)

          Review request for hbase and Ted Yu.

          Summary

          -------

          HBASE-2214 per scan max buffersize.

          This addresses bug HBASE-2214.

          https://issues.apache.org/jira/browse/HBASE-2214

          Diffs

          -----

          /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330680

          /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1330680

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1330680

          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1330680

          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1330680

          /src/main/protobuf/Client.proto 1330680

          /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1330680

          Diff: https://reviews.apache.org/r/4726/diff

          Testing

          -------

          It works when running this test:

          new HBaseTestingUtility(conf).startMiniCluster();

          HBaseAdmin admin = new HBaseAdmin(conf);

          if (!admin.tableExists("test")) { bq. HTableDescriptor tableDesc = new HTableDescriptor("test"); bq. tableDesc.addFamily(new HColumnDescriptor("fam")); bq. admin.createTable(tableDesc); bq. }

          HTable table = new HTable(conf, "test");

          Put put;

          put = new Put(Bytes.toBytes("row1"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1"));

          table.put(put);

          put = new Put(Bytes.toBytes("row2"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2"));

          table.put(put);

          put = new Put(Bytes.toBytes("row3"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3"));

          table.put(put);

          table.flushCommits();

          { bq. System.out.println("returns all rows at once because of the caching"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. }

          { bq. System.out.println("returns one row at a time because of the maxResultSize"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. scan.setMaxResultSize(1); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. }

          See output:

          returns all rows at once because of the caching

          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''

          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''

          2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3

          2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,}

          returns one row at a time because of the maxResultSize

          2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''

          2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''

          2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1

          2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1

          2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1

          Thanks,

          ferdy

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/#review7383 ----------------------------------------------------------- Where are we checking the size of the result made so far? I don't see it in the below. I'd expect it inside in the RegionScanner. Any chance of a test? Otherwise, patch looks great. /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java < https://reviews.apache.org/r/4726/#comment16293 > Is this going to be annoying? If a high-traffic server, won't this get logged once per request? Perhaps thousands a second? /src/main/java/org/apache/hadoop/hbase/client/Scan.java < https://reviews.apache.org/r/4726/#comment16294 > Is this needed? Is this set on Scan creation? When would it change after Scan construction? Or, are we using builder pattern here and its set after construction but before use? /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java < https://reviews.apache.org/r/4726/#comment16295 > oh, I see how its used now. ignore above comment. Michael On 2012-04-26 08:18:40, ferdy wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/ ----------------------------------------------------------- (Updated 2012-04-26 08:18:40) Review request for hbase and Ted Yu. Summary ------- HBASE-2214 per scan max buffersize. This addresses bug HBASE-2214 . https://issues.apache.org/jira/browse/HBASE-2214 Diffs ----- /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330680 /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1330680 /src/main/protobuf/Client.proto 1330680 /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1330680 Diff: https://reviews.apache.org/r/4726/diff Testing ------- It works when running this test: new HBaseTestingUtility(conf).startMiniCluster(); HBaseAdmin admin = new HBaseAdmin(conf); if (!admin.tableExists("test")) { bq. HTableDescriptor tableDesc = new HTableDescriptor("test"); bq. tableDesc.addFamily(new HColumnDescriptor("fam")); bq. admin.createTable(tableDesc); bq. } HTable table = new HTable(conf, "test"); Put put; put = new Put(Bytes.toBytes("row1")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1")); table.put(put); put = new Put(Bytes.toBytes("row2")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2")); table.put(put); put = new Put(Bytes.toBytes("row3")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3")); table.put(put); table.flushCommits(); { bq. System.out.println("returns all rows at once because of the caching"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. } { bq. System.out.println("returns one row at a time because of the maxResultSize"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. scan.setMaxResultSize(1); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. } See output: returns all rows at once because of the caching 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3 2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,} returns one row at a time because of the maxResultSize 2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1 Thanks, ferdy
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4726/
          -----------------------------------------------------------

          (Updated 2012-05-01 07:50:07.133923)

          Review request for hbase and Ted Yu.

          Changes
          -------

          v5 (updated to head and removed debug "rows returned" statement)

          Summary
          -------

          HBASE-2214 per scan max buffersize.

          This addresses bug HBASE-2214.
          https://issues.apache.org/jira/browse/HBASE-2214

          Diffs (updated)


          /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330680
          /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330680
          /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330680
          /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330680
          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330680
          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330680
          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330680
          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330680
          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330680
          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1332544
          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1332544
          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1332544
          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1332544
          /src/main/protobuf/Client.proto 1330680
          /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1332544

          Diff: https://reviews.apache.org/r/4726/diff

          Testing (updated)
          -------

          It works when running this test:

          new HBaseTestingUtility(conf).startMiniCluster();

          HBaseAdmin admin = new HBaseAdmin(conf);
          if (!admin.tableExists("test"))

          { HTableDescriptor tableDesc = new HTableDescriptor("test"); tableDesc.addFamily(new HColumnDescriptor("fam")); admin.createTable(tableDesc); }

          HTable table = new HTable(conf, "test");
          Put put;

          put = new Put(Bytes.toBytes("row1"));
          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1"));
          table.put(put);

          put = new Put(Bytes.toBytes("row2"));
          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2"));
          table.put(put);

          put = new Put(Bytes.toBytes("row3"));
          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3"));
          table.put(put);

          table.flushCommits();
          //put a logging statement to ClientScanner#next() to see the effect.

          { System.out.println("returns all rows at once because of the caching"); Scan scan = new Scan(); scan.setCaching(100); ResultScanner scanner = table.getScanner(scan); scanner.next(100); } { System.out.println("returns one row at a time because of the maxResultSize"); Scan scan = new Scan(); scan.setCaching(100); scan.setMaxResultSize(1); ResultScanner scanner = table.getScanner(scan); scanner.next(100); }

          See output:

          returns all rows at once because of the caching
          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''
          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''
          2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3
          2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at

          {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,}

          returns one row at a time because of the maxResultSize
          2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''
          2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''
          2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1
          2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1
          2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1

          Thanks,

          ferdy

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/ ----------------------------------------------------------- (Updated 2012-05-01 07:50:07.133923) Review request for hbase and Ted Yu. Changes ------- v5 (updated to head and removed debug "rows returned" statement) Summary ------- HBASE-2214 per scan max buffersize. This addresses bug HBASE-2214 . https://issues.apache.org/jira/browse/HBASE-2214 Diffs (updated) /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330680 /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1332544 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1332544 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1332544 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1332544 /src/main/protobuf/Client.proto 1330680 /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1332544 Diff: https://reviews.apache.org/r/4726/diff Testing (updated) ------- It works when running this test: new HBaseTestingUtility(conf).startMiniCluster(); HBaseAdmin admin = new HBaseAdmin(conf); if (!admin.tableExists("test")) { HTableDescriptor tableDesc = new HTableDescriptor("test"); tableDesc.addFamily(new HColumnDescriptor("fam")); admin.createTable(tableDesc); } HTable table = new HTable(conf, "test"); Put put; put = new Put(Bytes.toBytes("row1")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1")); table.put(put); put = new Put(Bytes.toBytes("row2")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2")); table.put(put); put = new Put(Bytes.toBytes("row3")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3")); table.put(put); table.flushCommits(); //put a logging statement to ClientScanner#next() to see the effect. { System.out.println("returns all rows at once because of the caching"); Scan scan = new Scan(); scan.setCaching(100); ResultScanner scanner = table.getScanner(scan); scanner.next(100); } { System.out.println("returns one row at a time because of the maxResultSize"); Scan scan = new Scan(); scan.setCaching(100); scan.setMaxResultSize(1); ResultScanner scanner = table.getScanner(scan); scanner.next(100); } See output: returns all rows at once because of the caching 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3 2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,} returns one row at a time because of the maxResultSize 2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1 Thanks, ferdy
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2012-04-30 20:03:34, Michael Stack wrote:

          > Where are we checking the size of the result made so far? I don't see it in the below. I'd expect it inside in the RegionScanner. Any chance of a test? Otherwise, patch looks great.

          Please see the testing method in the Testing Done field above. (Not sure where to add a test in the project).

          Thanks for the feedback.

          On 2012-04-30 20:03:34, Michael Stack wrote:

          > /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java, line 323

          > <https://reviews.apache.org/r/4726/diff/4/?file=104531#file104531line323>

          >

          > Is this going to be annoying? If a high-traffic server, won't this get logged once per request? Perhaps thousands a second?

          I accidently left this in when I uploaded new version of patch. It is removed now.

          • ferdy

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4726/#review7383
          -----------------------------------------------------------

          On 2012-05-01 07:50:07, ferdy wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/4726/

          -----------------------------------------------------------

          (Updated 2012-05-01 07:50:07)

          Review request for hbase and Ted Yu.

          Summary

          -------

          HBASE-2214 per scan max buffersize.

          This addresses bug HBASE-2214.

          https://issues.apache.org/jira/browse/HBASE-2214

          Diffs

          -----

          /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330680

          /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1332544

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1332544

          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1332544

          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1332544

          /src/main/protobuf/Client.proto 1330680

          /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1332544

          Diff: https://reviews.apache.org/r/4726/diff

          Testing

          -------

          It works when running this test:

          new HBaseTestingUtility(conf).startMiniCluster();

          HBaseAdmin admin = new HBaseAdmin(conf);

          if (!admin.tableExists("test")) { bq. HTableDescriptor tableDesc = new HTableDescriptor("test"); bq. tableDesc.addFamily(new HColumnDescriptor("fam")); bq. admin.createTable(tableDesc); bq. }

          HTable table = new HTable(conf, "test");

          Put put;

          put = new Put(Bytes.toBytes("row1"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1"));

          table.put(put);

          put = new Put(Bytes.toBytes("row2"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2"));

          table.put(put);

          put = new Put(Bytes.toBytes("row3"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3"));

          table.put(put);

          table.flushCommits();

          //put a logging statement to ClientScanner#next() to see the effect.

          { bq. System.out.println("returns all rows at once because of the caching"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. }

          { bq. System.out.println("returns one row at a time because of the maxResultSize"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. scan.setMaxResultSize(1); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. }

          See output:

          returns all rows at once because of the caching

          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''

          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''

          2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3

          2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,}

          returns one row at a time because of the maxResultSize

          2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''

          2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''

          2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1

          2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1

          2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1

          Thanks,

          ferdy

          Show
          jiraposter@reviews.apache.org added a comment - On 2012-04-30 20:03:34, Michael Stack wrote: > Where are we checking the size of the result made so far? I don't see it in the below. I'd expect it inside in the RegionScanner. Any chance of a test? Otherwise, patch looks great. Please see the testing method in the Testing Done field above. (Not sure where to add a test in the project). Thanks for the feedback. On 2012-04-30 20:03:34, Michael Stack wrote: > /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java, line 323 > < https://reviews.apache.org/r/4726/diff/4/?file=104531#file104531line323 > > > Is this going to be annoying? If a high-traffic server, won't this get logged once per request? Perhaps thousands a second? I accidently left this in when I uploaded new version of patch. It is removed now. ferdy ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/#review7383 ----------------------------------------------------------- On 2012-05-01 07:50:07, ferdy wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/ ----------------------------------------------------------- (Updated 2012-05-01 07:50:07) Review request for hbase and Ted Yu. Summary ------- HBASE-2214 per scan max buffersize. This addresses bug HBASE-2214 . https://issues.apache.org/jira/browse/HBASE-2214 Diffs ----- /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330680 /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1332544 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1332544 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1332544 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1332544 /src/main/protobuf/Client.proto 1330680 /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1332544 Diff: https://reviews.apache.org/r/4726/diff Testing ------- It works when running this test: new HBaseTestingUtility(conf).startMiniCluster(); HBaseAdmin admin = new HBaseAdmin(conf); if (!admin.tableExists("test")) { bq. HTableDescriptor tableDesc = new HTableDescriptor("test"); bq. tableDesc.addFamily(new HColumnDescriptor("fam")); bq. admin.createTable(tableDesc); bq. } HTable table = new HTable(conf, "test"); Put put; put = new Put(Bytes.toBytes("row1")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1")); table.put(put); put = new Put(Bytes.toBytes("row2")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2")); table.put(put); put = new Put(Bytes.toBytes("row3")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3")); table.put(put); table.flushCommits(); //put a logging statement to ClientScanner#next() to see the effect. { bq. System.out.println("returns all rows at once because of the caching"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. } { bq. System.out.println("returns one row at a time because of the maxResultSize"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. scan.setMaxResultSize(1); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. } See output: returns all rows at once because of the caching 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3 2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,} returns one row at a time because of the maxResultSize 2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1 Thanks, ferdy
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2012-04-30 20:03:34, Michael Stack wrote:

          > Where are we checking the size of the result made so far? I don't see it in the below. I'd expect it inside in the RegionScanner. Any chance of a test? Otherwise, patch looks great.

          ferdy wrote:

          Please see the testing method in the Testing Done field above. (Not sure where to add a test in the project).

          Thanks for the feedback.

          (One more thing, it works because of the previous work done in HBASE-1996. But it's arguably not the best way to implement it.)

          • ferdy

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4726/#review7383
          -----------------------------------------------------------

          On 2012-05-01 07:50:07, ferdy wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/4726/

          -----------------------------------------------------------

          (Updated 2012-05-01 07:50:07)

          Review request for hbase and Ted Yu.

          Summary

          -------

          HBASE-2214 per scan max buffersize.

          This addresses bug HBASE-2214.

          https://issues.apache.org/jira/browse/HBASE-2214

          Diffs

          -----

          /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330680

          /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330680

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1332544

          /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1332544

          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1332544

          /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1332544

          /src/main/protobuf/Client.proto 1330680

          /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1332544

          Diff: https://reviews.apache.org/r/4726/diff

          Testing

          -------

          It works when running this test:

          new HBaseTestingUtility(conf).startMiniCluster();

          HBaseAdmin admin = new HBaseAdmin(conf);

          if (!admin.tableExists("test")) { bq. HTableDescriptor tableDesc = new HTableDescriptor("test"); bq. tableDesc.addFamily(new HColumnDescriptor("fam")); bq. admin.createTable(tableDesc); bq. }

          HTable table = new HTable(conf, "test");

          Put put;

          put = new Put(Bytes.toBytes("row1"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1"));

          table.put(put);

          put = new Put(Bytes.toBytes("row2"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2"));

          table.put(put);

          put = new Put(Bytes.toBytes("row3"));

          put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3"));

          table.put(put);

          table.flushCommits();

          //put a logging statement to ClientScanner#next() to see the effect.

          { bq. System.out.println("returns all rows at once because of the caching"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. }

          { bq. System.out.println("returns one row at a time because of the maxResultSize"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. scan.setMaxResultSize(1); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. }

          See output:

          returns all rows at once because of the caching

          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''

          2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''

          2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3

          2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,}

          returns one row at a time because of the maxResultSize

          2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key ''

          2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at ''

          2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1

          2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1

          2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1

          Thanks,

          ferdy

          Show
          jiraposter@reviews.apache.org added a comment - On 2012-04-30 20:03:34, Michael Stack wrote: > Where are we checking the size of the result made so far? I don't see it in the below. I'd expect it inside in the RegionScanner. Any chance of a test? Otherwise, patch looks great. ferdy wrote: Please see the testing method in the Testing Done field above. (Not sure where to add a test in the project). Thanks for the feedback. (One more thing, it works because of the previous work done in HBASE-1996 . But it's arguably not the best way to implement it.) ferdy ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/#review7383 ----------------------------------------------------------- On 2012-05-01 07:50:07, ferdy wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4726/ ----------------------------------------------------------- (Updated 2012-05-01 07:50:07) Review request for hbase and Ted Yu. Summary ------- HBASE-2214 per scan max buffersize. This addresses bug HBASE-2214 . https://issues.apache.org/jira/browse/HBASE-2214 Diffs ----- /src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 1330680 /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 1330680 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1332544 /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1332544 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 1332544 /src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 1332544 /src/main/protobuf/Client.proto 1330680 /src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 1332544 Diff: https://reviews.apache.org/r/4726/diff Testing ------- It works when running this test: new HBaseTestingUtility(conf).startMiniCluster(); HBaseAdmin admin = new HBaseAdmin(conf); if (!admin.tableExists("test")) { bq. HTableDescriptor tableDesc = new HTableDescriptor("test"); bq. tableDesc.addFamily(new HColumnDescriptor("fam")); bq. admin.createTable(tableDesc); bq. } HTable table = new HTable(conf, "test"); Put put; put = new Put(Bytes.toBytes("row1")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual1"),Bytes.toBytes("val1")); table.put(put); put = new Put(Bytes.toBytes("row2")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual2"),Bytes.toBytes("val2")); table.put(put); put = new Put(Bytes.toBytes("row3")); put.add(Bytes.toBytes("fam"),Bytes.toBytes("qual3"),Bytes.toBytes("val3")); table.put(put); table.flushCommits(); //put a logging statement to ClientScanner#next() to see the effect. { bq. System.out.println("returns all rows at once because of the caching"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. } { bq. System.out.println("returns one row at a time because of the maxResultSize"); bq. Scan scan = new Scan(); bq. scan.setCaching(100); bq. scan.setMaxResultSize(1); bq. ResultScanner scanner = table.getScanner(scan); bq. scanner.next(100); bq. } See output: returns all rows at once because of the caching 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,494 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,499 DEBUG [main] client.ClientScanner(323): Rows returned 3 2012-04-25 22:18:47,502 DEBUG [main] client.ClientScanner(193): Finished with scanning at {NAME => 'test,,1335385126388.ed23a82f3d6ca2eab571918843796259.', STARTKEY => '', ENDKEY => '', ENCODED => ed23a82f3d6ca2eab571918843796259,} returns one row at a time because of the maxResultSize 2012-04-25 22:18:47,504 DEBUG [main] client.ClientScanner(94): Creating scanner over test starting at key '' 2012-04-25 22:18:47,505 DEBUG [main] client.ClientScanner(206): Advancing internal scanner to startKey at '' 2012-04-25 22:18:47,514 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,517 DEBUG [main] client.ClientScanner(323): Rows returned 1 2012-04-25 22:18:47,522 DEBUG [main] client.ClientScanner(323): Rows returned 1 Thanks, ferdy
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12525160/HBASE-2214-v6.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 hadoop23. The patch compiles against the hadoop 0.23.x profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.client.TestShell
          org.apache.hadoop.hbase.master.TestMasterNoCluster

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1704//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1704//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1704//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525160/HBASE-2214-v6.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestShell org.apache.hadoop.hbase.master.TestMasterNoCluster Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1704//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1704//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1704//console This message is automatically generated.
          Hide
          stack added a comment -

          +1 on patch . Mind rebasing to trunk Ferdy so I can apply? Thanks boss.

          Show
          stack added a comment - +1 on patch . Mind rebasing to trunk Ferdy so I can apply? Thanks boss.
          Hide
          Ferdy Galema added a comment -

          v7 of patch (changes are because of making patch work on latest trunk)

          Show
          Ferdy Galema added a comment - v7 of patch (changes are because of making patch work on latest trunk)
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12525267/HBASE-2214-v7.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 hadoop23. The patch compiles against the hadoop 0.23.x profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1719//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1719//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1719//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525267/HBASE-2214-v7.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1719//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1719//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1719//console This message is automatically generated.
          Hide
          Ted Yu added a comment -

          Will integrate the patch later today if there is no objection.

          Show
          Ted Yu added a comment - Will integrate the patch later today if there is no objection.
          Hide
          Ted Yu added a comment -

          Integrated to trunk.

          Waiting for updated patch for 0.94.

          Show
          Ted Yu added a comment - Integrated to trunk. Waiting for updated patch for 0.94.
          Hide
          Ferdy Galema added a comment -

          Here's the patch for 0.94. It's tested with the same method as described in the patch review. I can verify it works.

          Show
          Ferdy Galema added a comment - Here's the patch for 0.94. It's tested with the same method as described in the patch review. I can verify it works.
          Hide
          Ferdy Galema added a comment -

          Hold a sec while I upload one without the call to system.out

          Show
          Ferdy Galema added a comment - Hold a sec while I upload one without the call to system.out
          Hide
          Ferdy Galema added a comment -

          Done.

          Show
          Ferdy Galema added a comment - Done.
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #2837 (See https://builds.apache.org/job/HBase-TRUNK/2837/)
          HBASE-2214 Do HBASE-1996 – setting size to return in scan rather than count of rows – properly (Ferdy Galema) (Revision 1333122)

          Result = SUCCESS
          tedyu :
          Files :

          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Scan.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java
          • /hbase/trunk/src/main/protobuf/Client.proto
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK #2837 (See https://builds.apache.org/job/HBase-TRUNK/2837/ ) HBASE-2214 Do HBASE-1996 – setting size to return in scan rather than count of rows – properly (Ferdy Galema) (Revision 1333122) Result = SUCCESS tedyu : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Scan.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java /hbase/trunk/src/main/protobuf/Client.proto /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
          Hide
          Ted Yu added a comment -

          Integrated HBASE-2214-0.94-v3.txt to 0.94 branch.

          Thanks for the patch, Ferdy.

          Thanks for the review, Stack.

          Show
          Ted Yu added a comment - Integrated HBASE-2214 -0.94-v3.txt to 0.94 branch. Thanks for the patch, Ferdy. Thanks for the review, Stack.
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94 #170 (See https://builds.apache.org/job/HBase-0.94/170/)
          HBASE-2214 Do HBASE-1996 – setting size to return in scan rather than count of rows – properly (Ferdy Galema) (Revision 1333157)

          Result = FAILURE
          tedyu :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/Scan.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
          Show
          Hudson added a comment - Integrated in HBase-0.94 #170 (See https://builds.apache.org/job/HBase-0.94/170/ ) HBASE-2214 Do HBASE-1996 – setting size to return in scan rather than count of rows – properly (Ferdy Galema) (Revision 1333157) Result = FAILURE tedyu : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/Scan.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK-security #190 (See https://builds.apache.org/job/HBase-TRUNK-security/190/)
          HBASE-2214 Do HBASE-1996 – setting size to return in scan rather than count of rows – properly (Ferdy Galema) (Revision 1333122)

          Result = SUCCESS
          tedyu :
          Files :

          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Scan.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java
          • /hbase/trunk/src/main/protobuf/Client.proto
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK-security #190 (See https://builds.apache.org/job/HBase-TRUNK-security/190/ ) HBASE-2214 Do HBASE-1996 – setting size to return in scan rather than count of rows – properly (Ferdy Galema) (Revision 1333122) Result = SUCCESS tedyu : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Scan.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java /hbase/trunk/src/main/protobuf/Client.proto /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94-security #26 (See https://builds.apache.org/job/HBase-0.94-security/26/)
          HBASE-2214 Do HBASE-1996 – setting size to return in scan rather than count of rows – properly (Ferdy Galema) (Revision 1333157)

          Result = SUCCESS
          tedyu :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/Scan.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
          Show
          Hudson added a comment - Integrated in HBase-0.94-security #26 (See https://builds.apache.org/job/HBase-0.94-security/26/ ) HBASE-2214 Do HBASE-1996 – setting size to return in scan rather than count of rows – properly (Ferdy Galema) (Revision 1333157) Result = SUCCESS tedyu : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/Scan.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
          Hide
          stack added a comment -

          This was committed a while back.

          Show
          stack added a comment - This was committed a while back.
          Hide
          Lars Hofhansl added a comment -

          This changes the Scan version which causes compatibility issues with 0.92.
          It should be rolled back from 0.94.

          Show
          Lars Hofhansl added a comment - This changes the Scan version which causes compatibility issues with 0.92. It should be rolled back from 0.94.
          Hide
          Lars Hofhansl added a comment -

          In fact this even break compatibility for 0.94.0 (which we just ran into at Salesforce).
          Rolling back from 0.94.1 now.

          Show
          Lars Hofhansl added a comment - In fact this even break compatibility for 0.94.0 (which we just ran into at Salesforce). Rolling back from 0.94.1 now.
          Hide
          Ted Yu added a comment -

          Rollback is Okay.
          Some code is needed to keep this feature and not breaking compatibility at the same time.

          For the moment, rollback is the easiest to do.

          Show
          Ted Yu added a comment - Rollback is Okay. Some code is needed to keep this feature and not breaking compatibility at the same time. For the moment, rollback is the easiest to do.
          Hide
          Lars Hofhansl added a comment -

          Reverted from 0.94.1. I pushed it to 0.94.2 so we can think about how to get the functionality without the version change to Scan.

          Show
          Lars Hofhansl added a comment - Reverted from 0.94.1. I pushed it to 0.94.2 so we can think about how to get the functionality without the version change to Scan.
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94-security #45 (See https://builds.apache.org/job/HBase-0.94-security/45/)
          HBASE-2214 Revert - breaks compatibility with 0.92.x and 0.94.0 (Revision 1365204)

          Result = FAILURE
          larsh :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/Scan.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
          Show
          Hudson added a comment - Integrated in HBase-0.94-security #45 (See https://builds.apache.org/job/HBase-0.94-security/45/ ) HBASE-2214 Revert - breaks compatibility with 0.92.x and 0.94.0 (Revision 1365204) Result = FAILURE larsh : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/Scan.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94 #367 (See https://builds.apache.org/job/HBase-0.94/367/)
          HBASE-2214 Revert - breaks compatibility with 0.92.x and 0.94.0 (Revision 1365204)

          Result = SUCCESS
          larsh :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/Scan.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
          Show
          Hudson added a comment - Integrated in HBase-0.94 #367 (See https://builds.apache.org/job/HBase-0.94/367/ ) HBASE-2214 Revert - breaks compatibility with 0.92.x and 0.94.0 (Revision 1365204) Result = SUCCESS larsh : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/Scan.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
          Hide
          Ferdy Galema added a comment -

          How can this break compatibily? (I would say that the Scan version number actually maintains compatibility). Anyway, I think it would be best if someone else picked this up. That will be much faster to implement it correctly. (As I'm not involved enough).

          Show
          Ferdy Galema added a comment - How can this break compatibily? (I would say that the Scan version number actually maintains compatibility). Anyway, I think it would be best if someone else picked this up. That will be much faster to implement it correctly. (As I'm not involved enough).
          Hide
          Lars Hofhansl added a comment -

          Hi Ferdy, we definitely appreciate all the work you put in for this patch! It is still in 0.96. We'll try to find a way to use your patch without changing the version number.

          The problem with changing the version number (in 0.94 and prior) is that then the newer version of the object cannot be deserialized by an older version.
          You can see this Scan.readFields(), which throws an exception if it reads a version newer than the one it expects.
          In practice this means that now a new client cannot issue a scan request to an older server, because the server cannot deserialize the scan object.

          In 0.96 and later these kind of problems are solved by protobufs.

          Show
          Lars Hofhansl added a comment - Hi Ferdy, we definitely appreciate all the work you put in for this patch! It is still in 0.96. We'll try to find a way to use your patch without changing the version number. The problem with changing the version number (in 0.94 and prior) is that then the newer version of the object cannot be deserialized by an older version. You can see this Scan.readFields(), which throws an exception if it reads a version newer than the one it expects. In practice this means that now a new client cannot issue a scan request to an older server, because the server cannot deserialize the scan object. In 0.96 and later these kind of problems are solved by protobufs.
          Hide
          Ferdy Galema added a comment -

          Thanks Lars.

          Show
          Ferdy Galema added a comment - Thanks Lars.
          Hide
          Ted Yu added a comment -

          Scan class should be flexible in serializing scan version.
          When the older server throws exception, Scan object can downgrade to earlier scan version and retry.

          Show
          Ted Yu added a comment - Scan class should be flexible in serializing scan version. When the older server throws exception, Scan object can downgrade to earlier scan version and retry.
          Hide
          Lars Hofhansl added a comment -

          Yep. Scan can check the version in readFields and then act with a bit more smarts than just throwing up its hands (an exception).

          Show
          Lars Hofhansl added a comment - Yep. Scan can check the version in readFields and then act with a bit more smarts than just throwing up its hands (an exception).
          Hide
          Jean-Daniel Cryans added a comment -

          Following this discussion on user@

          http://permalink.gmane.org/gmane.comp.java.hadoop.hbase.user/27936

          If the max scanner result size is only set on the region server it's possible miss rows. Even with the latest patch in this jira it's still possible since the client doesn't tell the server it's going to use the default instead of whatever the server is using.

          Maybe the ClientScanner could modify the Scan that it passes the max scanner result it uses to the region server

          Show
          Jean-Daniel Cryans added a comment - Following this discussion on user@ http://permalink.gmane.org/gmane.comp.java.hadoop.hbase.user/27936 If the max scanner result size is only set on the region server it's possible miss rows. Even with the latest patch in this jira it's still possible since the client doesn't tell the server it's going to use the default instead of whatever the server is using. Maybe the ClientScanner could modify the Scan that it passes the max scanner result it uses to the region server
          Hide
          Lars Hofhansl added a comment -

          @Ferdy, are you sure you do not want to work on this anymore?
          I'll change Scan.java to be able to deal with two different version, so we can get this in 0.94 (admittedly, that is a small change only).

          Maybe the ClientScanner could modify the Scan that it passes the max scanner result it uses to the region server

          Of course the requires the server to actually trust the client. Although I think with a rogue client it is quite easy to kill/mess-with a RegionServer in many other areas.

          Show
          Lars Hofhansl added a comment - @Ferdy, are you sure you do not want to work on this anymore? I'll change Scan.java to be able to deal with two different version, so we can get this in 0.94 (admittedly, that is a small change only). Maybe the ClientScanner could modify the Scan that it passes the max scanner result it uses to the region server Of course the requires the server to actually trust the client. Although I think with a rogue client it is quite easy to kill/mess-with a RegionServer in many other areas.
          Hide
          Jean-Daniel Cryans added a comment -

          Of course the requires the server to actually trust the client.

          Isn't that what this jira does already?

          Show
          Jean-Daniel Cryans added a comment - Of course the requires the server to actually trust the client. Isn't that what this jira does already?
          Hide
          Lars Hofhansl added a comment -

          Yep... Was just making a general observation.

          Show
          Lars Hofhansl added a comment - Yep... Was just making a general observation.
          Hide
          Ferdy Galema added a comment -

          I'm happy with you or anyone else wrapping this up. (I'm not sure if the client/server buffersize mismatch issue deserves an issue on its own or if you guys want to solve it in this issue. That's up to you.)

          Show
          Ferdy Galema added a comment - I'm happy with you or anyone else wrapping this up. (I'm not sure if the client/server buffersize mismatch issue deserves an issue on its own or if you guys want to solve it in this issue. That's up to you.)
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94-security-on-Hadoop-23 #6 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/6/)
          HBASE-2214 Revert - breaks compatibility with 0.92.x and 0.94.0 (Revision 1365204)

          Result = FAILURE
          larsh :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/Scan.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
          Show
          Hudson added a comment - Integrated in HBase-0.94-security-on-Hadoop-23 #6 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/6/ ) HBASE-2214 Revert - breaks compatibility with 0.92.x and 0.94.0 (Revision 1365204) Result = FAILURE larsh : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/Scan.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
          Hide
          Lars Hofhansl added a comment -

          Actually in order to make this work, we'd have make sure an older server can read the Scan object from newer client, so we cannot do this (since an old servers are deployed already).

          If there is no way to do this without changing the version of the Scan version I suggest we defer this until 0.96.

          Show
          Lars Hofhansl added a comment - Actually in order to make this work, we'd have make sure an older server can read the Scan object from newer client, so we cannot do this (since an old servers are deployed already). If there is no way to do this without changing the version of the Scan version I suggest we defer this until 0.96.
          Hide
          Lars Hofhansl added a comment -

          Removing 0.94 target after all

          Show
          Lars Hofhansl added a comment - Removing 0.94 target after all
          Hide
          stack added a comment -

          Marking closed.

          Show
          stack added a comment - Marking closed.

            People

            • Assignee:
              Unassigned
              Reporter:
              stack
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development