Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.6, 0.94.7
    • Fix Version/s: 0.98.0
    • Component/s: Client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Do a reversed client scan by setting 'reversed' as true in Scan.Java.
    • Tags:
      Phoenix

      Description

      Reversed scan means scan the rows backward.
      And StartRow bigger than StopRow in a reversed scan.

      For example, for the following rows:

      aaa/c1:q1/value1
      aaa/c1:q2/value2
      bbb/c1:q1/value1
      bbb/c1:q2/value2
      ccc/c1:q1/value1
      ccc/c1:q2/value2
      ddd/c1:q1/value1
      ddd/c1:q2/value2
      eee/c1:q1/value1
      eee/c1:q2/value2

      you could do a reversed scan from 'ddd' to 'bbb'(exclude) like this:

      Scan scan = new Scan();
      scan.setStartRow('ddd');
      scan.setStopRow('bbb');
      scan.setReversed(true);
      for(Result result:htable.getScanner(scan))

      { System.out.println(result); }

      Aslo you could do the reversed scan with shell like this:
      hbase> scan 'table',

      {REVERSED => true,STARTROW=>'ddd', STOPROW=>'bbb'}

      And the output is:
      ddd/c1:q1/value1
      ddd/c1:q2/value2
      ccc/c1:q1/value1
      ccc/c1:q2/value2

      All the documentation I find about HBase says that if you want forward and reverse scans you should just build 2 tables and one be ascending and one descending. Is there a fundamental reason that HBase only supports forward Scan? It seems like a lot of extra space overhead and coding overhead (to keep them in sync) to support 2 tables.

      I am assuming this has been discussed before, but I can't find the discussions anywhere about it or why it would be infeasible.

      1. hbase-4811-trunkv9.patch
        138 kB
        chunhui shen
      2. hbase-4811-trunkv8.patch
        138 kB
        chunhui shen
      3. hbase-4811-trunkv7.patch
        136 kB
        chunhui shen
      4. hbase-4811-trunkv6.patch
        136 kB
        chunhui shen
      5. hbase-4811-trunkv4.patch
        135 kB
        chunhui shen
      6. hbase-4811-trunkv28.patch
        153 kB
        chunhui shen
      7. hbase-4811-trunkv27.patch
        162 kB
        chunhui shen
      8. hbase-4811-trunkv26.patch
        162 kB
        chunhui shen
      9. hbase-4811-trunkv25.patch
        162 kB
        chunhui shen
      10. hbase-4811-trunkv24.patch
        162 kB
        chunhui shen
      11. hbase-4811-trunkv24.patch
        162 kB
        chunhui shen
      12. hbase-4811-trunkv21.patch
        161 kB
        chunhui shen
      13. hbase-4811-trunkv20.patch
        163 kB
        chunhui shen
      14. hbase-4811-trunkv19.patch
        153 kB
        chunhui shen
      15. hbase-4811-trunkv18.patch
        152 kB
        chunhui shen
      16. hbase-4811-trunkv17.patch
        152 kB
        chunhui shen
      17. hbase-4811-trunkv16.patch
        152 kB
        chunhui shen
      18. hbase-4811-trunkv15.patch
        152 kB
        chunhui shen
      19. hbase-4811-trunkv14.patch
        145 kB
        chunhui shen
      20. hbase-4811-trunkv13.patch
        152 kB
        chunhui shen
      21. hbase-4811-trunkv12.patch
        151 kB
        chunhui shen
      22. hbase-4811-trunkv11.patch
        139 kB
        chunhui shen
      23. hbase-4811-trunkv1.patch
        133 kB
        chunhui shen
      24. hbase-4811-0.94-v24.patch
        147 kB
        chunhui shen
      25. hbase-4811-0.94 v21.patch
        147 kB
        chunhui shen
      26. HBase-4811-0.94-v2.txt
        93 kB
        Liang Xie
      27. HBase-4811-0.94.3modified.txt
        55 kB
        Liang Xie
      28. 4811-trunk-v5.patch
        135 kB
        Ted Yu
      29. 4811-trunk-v29.patch
        153 kB
        Ted Yu
      30. 4811-trunk-v10.txt
        138 kB
        Ted Yu
      31. 4811-0.94-v3.txt
        91 kB
        Lars Hofhansl
      32. 4811-0.94-v25.txt
        147 kB
        Lars Hofhansl
      33. 4811-0.94-v23.txt
        151 kB
        Lars Hofhansl
      34. 4811-0.94-v22.txt
        147 kB
        Lars Hofhansl

        Issue Links

          Activity

          Hide
          stack added a comment -

          Is there a fundamental reason that HBase only supports forward Scan?

          Yes. All the data is sorted in one direction only and all Scan objects are written to go in the data's 'natural' direction. There is no native support for going backwards whether its reading from files 'backwards' or getting a view on our MemStore that gives a reverse-sort-view.

          To make it work, you'd have to write a bunch of code and you'd be always going against the grain.

          It used to come up the odd time in the early days but versions on the above args would usually quiet them.

          If you need more detail, ask.

          Show
          stack added a comment - Is there a fundamental reason that HBase only supports forward Scan? Yes. All the data is sorted in one direction only and all Scan objects are written to go in the data's 'natural' direction. There is no native support for going backwards whether its reading from files 'backwards' or getting a view on our MemStore that gives a reverse-sort-view. To make it work, you'd have to write a bunch of code and you'd be always going against the grain. It used to come up the odd time in the early days but versions on the above args would usually quiet them. If you need more detail, ask.
          Hide
          John Carrino added a comment -

          Yeah, I'm not that familiar with the codebase, but I'd assume that in
          order to get forward scans you'd have to have the data sorted. And
          from what I understand it is internally stored as "sstables" or
          HFiles. If you have it sorted to scan in one direction, it seems
          pretty easy to go the other direction. LevelDb uses ssTables and
          supports reverse ranges.

          The only thing that I could think of from the design (from a high
          level) that might make it difficult to do reverse ranges is dealing
          with splitting ranges when moving ranges from one region server to
          another.

          Just from a quick look at MemStore that you mention, it uses a
          KeyValueSkipListSet under the covers that is a NavigableSet and
          supports descendingSet and descendingIterator.

          Also to provide some context, this table we want to scan both ways is effectively an index which will be relatively small and we would like to pin in memory (as much as possible). Also likely that this will run on all Sold State, so doing reverse reads won't be a perf hit like it would be for spinny drives.

          Show
          John Carrino added a comment - Yeah, I'm not that familiar with the codebase, but I'd assume that in order to get forward scans you'd have to have the data sorted. And from what I understand it is internally stored as "sstables" or HFiles. If you have it sorted to scan in one direction, it seems pretty easy to go the other direction. LevelDb uses ssTables and supports reverse ranges. The only thing that I could think of from the design (from a high level) that might make it difficult to do reverse ranges is dealing with splitting ranges when moving ranges from one region server to another. Just from a quick look at MemStore that you mention, it uses a KeyValueSkipListSet under the covers that is a NavigableSet and supports descendingSet and descendingIterator. Also to provide some context, this table we want to scan both ways is effectively an index which will be relatively small and we would like to pin in memory (as much as possible). Also likely that this will run on all Sold State, so doing reverse reads won't be a perf hit like it would be for spinny drives.
          Hide
          stack added a comment -

          I'd suggest you spend more time w/ the code base to see how much of effort would be required doing a reverse scan (Superficially, yes, our MemStore is a NavigableSet but that is not what client interacts with; ditto our sstable-like hfile thing. IIRC leveldb counsels that the reverse range is going against the grain and at a minimum is much slower than the natural scan).

          Show
          stack added a comment - I'd suggest you spend more time w/ the code base to see how much of effort would be required doing a reverse scan (Superficially, yes, our MemStore is a NavigableSet but that is not what client interacts with; ditto our sstable-like hfile thing. IIRC leveldb counsels that the reverse range is going against the grain and at a minimum is much slower than the natural scan).
          Hide
          John Carrino added a comment -

          So looking at the region splitting code it looks like any scans that
          are open on ranges that are split get a special exception type and
          then just open a new scanner. So we don't have to worry about reverse
          iteration any more than forward with respect to splitting.

          I think this might boil down to writing a reverse iterator
          (HFileScanner) for HFile.

          -jc

          On Tue, Nov 22, 2011 at 3:48 PM, stack (Commented) (JIRA)

          Show
          John Carrino added a comment - So looking at the region splitting code it looks like any scans that are open on ranges that are split get a special exception type and then just open a new scanner. So we don't have to worry about reverse iteration any more than forward with respect to splitting. I think this might boil down to writing a reverse iterator (HFileScanner) for HFile. -jc On Tue, Nov 22, 2011 at 3:48 PM, stack (Commented) (JIRA)
          Hide
          John Carrino added a comment -

          Digging a littler deeper it appears that this was already planned when the V2 HFile format was written. In the header of a block is the offset of the previous block of the same type. I think this is currently used to support efficient lookups when seeking to a location, but could also be used easily for reverse scan.

          Show
          John Carrino added a comment - Digging a littler deeper it appears that this was already planned when the V2 HFile format was written. In the header of a block is the offset of the previous block of the same type. I think this is currently used to support efficient lookups when seeking to a location, but could also be used easily for reverse scan.
          Hide
          Liang Xie added a comment -

          Here is the perf number from leveldb's site: https://code.google.com/p/leveldb/

          We list the performance of reading sequentially in both the forward and reverse direction, and also the performance of a random lookup. Note that the database created by the benchmark is quite small. Therefore the report characterizes the performance of leveldb when the working set fits in memory. The cost of reading a piece of data that is not present in the operating system buffer cache will be dominated by the one or two disk seeks needed to fetch the data from disk. Write performance will be mostly unaffected by whether or not the working set fits in memory.

          readrandom : 16.677 micros/op; (approximately 60,000 reads per second)
          readseq : 0.476 micros/op; 232.3 MB/s
          readreverse : 0.724 micros/op; 152.9 MB/s

          LevelDB compacts its underlying storage data in the background to improve read performance. The results listed above were done immediately after a lot of random writes. The results after compactions (which are usually triggered automatically) are better.

          readrandom : 11.602 micros/op; (approximately 85,000 reads per second)
          readseq : 0.423 micros/op; 261.8 MB/s
          readreverse : 0.663 micros/op; 166.9 MB/s

          Show
          Liang Xie added a comment - Here is the perf number from leveldb's site: https://code.google.com/p/leveldb/ We list the performance of reading sequentially in both the forward and reverse direction, and also the performance of a random lookup. Note that the database created by the benchmark is quite small. Therefore the report characterizes the performance of leveldb when the working set fits in memory. The cost of reading a piece of data that is not present in the operating system buffer cache will be dominated by the one or two disk seeks needed to fetch the data from disk. Write performance will be mostly unaffected by whether or not the working set fits in memory. readrandom : 16.677 micros/op; (approximately 60,000 reads per second) readseq : 0.476 micros/op; 232.3 MB/s readreverse : 0.724 micros/op; 152.9 MB/s LevelDB compacts its underlying storage data in the background to improve read performance. The results listed above were done immediately after a lot of random writes. The results after compactions (which are usually triggered automatically) are better. readrandom : 11.602 micros/op; (approximately 85,000 reads per second) readseq : 0.423 micros/op; 261.8 MB/s readreverse : 0.663 micros/op; 166.9 MB/s
          Hide
          Liang Xie added a comment -

          seems reverse scan degrades read perf about 30%~40%, it's still acceptable per my view
          IIRC cassandra supports scan backwards as well.
          It would be better if HBase could provide this feature besides creating another mirror table.

          Show
          Liang Xie added a comment - seems reverse scan degrades read perf about 30%~40%, it's still acceptable per my view IIRC cassandra supports scan backwards as well. It would be better if HBase could provide this feature besides creating another mirror table.
          Hide
          John Carrino added a comment -

          A common workflow for me would to be to do a forward/reverse index lookup. Basically find the first row that is after/before a given key.

          I think for this case both forward and reverse index lookups would be approximately the same speed.

          Show
          John Carrino added a comment - A common workflow for me would to be to do a forward/reverse index lookup. Basically find the first row that is after/before a given key. I think for this case both forward and reverse index lookups would be approximately the same speed.
          Hide
          Liang Xie added a comment -

          Attached HBase-4811-0.94.3modified.txt is an our internal draft implementation for reverse scan. I thought it may be not suitable to merge into trunk or any branch, so i didn't make a patch against trunk or 0.94 branch, drop here just in case some guys are interesting on it, for reference only We didn't impletement it like leveldb&cassandra's way, which done in block layer, our policy is more focused on upper logic layer(scanner & Kv heap), though we still use seekBefore api.

          Performance: i ran PerformanceEvaluation, observed about 20%-40% degraded on scanRange10/100/1000, and approximately 50% degraded on randomSeekScan.

          Show
          Liang Xie added a comment - Attached HBase-4811-0.94.3modified.txt is an our internal draft implementation for reverse scan. I thought it may be not suitable to merge into trunk or any branch, so i didn't make a patch against trunk or 0.94 branch, drop here just in case some guys are interesting on it, for reference only We didn't impletement it like leveldb&cassandra's way, which done in block layer, our policy is more focused on upper logic layer(scanner & Kv heap), though we still use seekBefore api. Performance: i ran PerformanceEvaluation, observed about 20%-40% degraded on scanRange10/100/1000, and approximately 50% degraded on randomSeekScan.
          Hide
          John Carrino added a comment -

          That sounds great. One minor point is that I don't think that cassandra has reverse range scans across keys (even with the ordered partitioner), just reverse iteration for the column names within a specified row-name.

          Show
          John Carrino added a comment - That sounds great. One minor point is that I don't think that cassandra has reverse range scans across keys (even with the ordered partitioner), just reverse iteration for the column names within a specified row-name.
          Hide
          Liang Xie added a comment -

          Previous HBase-4811-0.94.3modified.txt didn't have a backward-compatibility, we can bump a SCAN version, like HBASE-6250's style:

          e.g.:

           
          --- main/java/org/apache/hadoop/hbase/client/Scan.java	(revision 88835)
          +++ main/java/org/apache/hadoop/hbase/client/Scan.java	(working copy)
          @@ -85,6 +85,7 @@
             private static final String ISOLATION_LEVEL = "_isolationlevel_";
           
             private static final byte SCAN_VERSION = (byte)2;
          +  private static final byte SCAN_REVERSED_VERSION = (byte)3;
             private byte [] startRow = HConstants.EMPTY_START_ROW;
             private byte [] stopRow  = HConstants.EMPTY_END_ROW;
             private int maxVersions = 1;
          @@ -577,7 +578,7 @@
             public void readFields(final DataInput in)
             throws IOException {
               int version = in.readByte();
          -    if (version > (int)SCAN_VERSION) {
          +    if (version > SCAN_REVERSED_VERSION) {
                 throw new IOException("version not supported");
               }
               this.startRow = Bytes.readByteArray(in);
          @@ -586,7 +587,9 @@
               this.batch = in.readInt();
               this.caching = in.readInt();
               this.cacheBlocks = in.readBoolean();
          -    this.reverse = in.readBoolean();
          +    if (version >= SCAN_REVERSED_VERSION) {
          +      this.reverse = in.readBoolean();
          +    }
               if(in.readBoolean()) {
                 this.filter = (Filter)createForName(Bytes.toString(Bytes.readByteArray(in)));
                 this.filter.readFields(in);
          @@ -614,14 +617,20 @@
           
             public void write(final DataOutput out)
             throws IOException {
          -    out.writeByte(SCAN_VERSION);
          +    if (reverse) {
          +      out.writeByte(SCAN_REVERSED_VERSION);
          +    } else {
          +      out.writeByte(SCAN_VERSION);
          +    }
               Bytes.writeByteArray(out, this.startRow);
               Bytes.writeByteArray(out, this.stopRow);
               out.writeInt(this.maxVersions);
               out.writeInt(this.batch);
               out.writeInt(this.caching);
               out.writeBoolean(this.cacheBlocks);
          -    out.writeBoolean(this.reverse);
          +    if (reverse) {    
          +      out.writeBoolean(this.reverse);
          +    }
               if(this.filter == null) {
                 out.writeBoolean(false);
               } else {
          
          Show
          Liang Xie added a comment - Previous HBase-4811-0.94.3modified.txt didn't have a backward-compatibility, we can bump a SCAN version, like HBASE-6250 's style: e.g.: --- main/java/org/apache/hadoop/hbase/client/Scan.java (revision 88835) +++ main/java/org/apache/hadoop/hbase/client/Scan.java (working copy) @@ -85,6 +85,7 @@ private static final String ISOLATION_LEVEL = "_isolationlevel_" ; private static final byte SCAN_VERSION = ( byte )2; + private static final byte SCAN_REVERSED_VERSION = ( byte )3; private byte [] startRow = HConstants.EMPTY_START_ROW; private byte [] stopRow = HConstants.EMPTY_END_ROW; private int maxVersions = 1; @@ -577,7 +578,7 @@ public void readFields( final DataInput in) throws IOException { int version = in.readByte(); - if (version > ( int )SCAN_VERSION) { + if (version > SCAN_REVERSED_VERSION) { throw new IOException( "version not supported" ); } this .startRow = Bytes.readByteArray(in); @@ -586,7 +587,9 @@ this .batch = in.readInt(); this .caching = in.readInt(); this .cacheBlocks = in.readBoolean(); - this .reverse = in.readBoolean(); + if (version >= SCAN_REVERSED_VERSION) { + this .reverse = in.readBoolean(); + } if (in.readBoolean()) { this .filter = (Filter)createForName(Bytes.toString(Bytes.readByteArray(in))); this .filter.readFields(in); @@ -614,14 +617,20 @@ public void write( final DataOutput out) throws IOException { - out.writeByte(SCAN_VERSION); + if (reverse) { + out.writeByte(SCAN_REVERSED_VERSION); + } else { + out.writeByte(SCAN_VERSION); + } Bytes.writeByteArray(out, this .startRow); Bytes.writeByteArray(out, this .stopRow); out.writeInt( this .maxVersions); out.writeInt( this .batch); out.writeInt( this .caching); out.writeBoolean( this .cacheBlocks); - out.writeBoolean( this .reverse); + if (reverse) { + out.writeBoolean( this .reverse); + } if ( this .filter == null ) { out.writeBoolean( false ); } else {
          Hide
          ramkrishna.s.vasudevan added a comment -

          I will check this patch today or tomorrow. Looks interesting.

          Show
          ramkrishna.s.vasudevan added a comment - I will check this patch today or tomorrow. Looks interesting.
          Hide
          Lars Hofhansl added a comment -

          If we wanted to have this backward compatible in 0.94 we would need to encode the reversed bit in the Scan's attributes... Or use a new scan version only reversed is true (so that an old regionserver would fail the scan request).

          Show
          Lars Hofhansl added a comment - If we wanted to have this backward compatible in 0.94 we would need to encode the reversed bit in the Scan's attributes... Or use a new scan version only reversed is true (so that an old regionserver would fail the scan request).
          Hide
          Liang Xie added a comment -

          right, Lars Hofhansl. we had introduced a new field in Scan and bumped a higher scan version in internal version before. And we fixed several bugs during adding more test cases. It's still unstable and we have a plan to improve and enable reverse scan in our product cluster. I'll give a updated patch if the production env shows great.

          Show
          Liang Xie added a comment - right, Lars Hofhansl . we had introduced a new field in Scan and bumped a higher scan version in internal version before. And we fixed several bugs during adding more test cases. It's still unstable and we have a plan to improve and enable reverse scan in our product cluster. I'll give a updated patch if the production env shows great.
          Hide
          Liang Xie added a comment -

          I can rebase a latest internal patch agaist 0.94 branch, not trunk, is that OK?

          Show
          Liang Xie added a comment - I can rebase a latest internal patch agaist 0.94 branch, not trunk, is that OK?
          Hide
          Liang Xie added a comment -

          a tentative rebase against 0.94 branch(it passed runAllTests in our internal 0.94.3 codebase)

          Show
          Liang Xie added a comment - a tentative rebase against 0.94 branch(it passed runAllTests in our internal 0.94.3 codebase)
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12584478/HBase-4811-0.94-v2.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 19 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5796//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12584478/HBase-4811-0.94-v2.txt against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 19 new or modified tests. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5796//console This message is automatically generated.
          Hide
          Liang Xie added a comment -

          repost

          Show
          Liang Xie added a comment - repost
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12584481/HBase-4811-0.94-v2.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 19 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5797//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12584481/HBase-4811-0.94-v2.txt against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 19 new or modified tests. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5797//console This message is automatically generated.
          Hide
          Lars Hofhansl added a comment -

          HadoopQA does not work against 0.94. Would need a trunk patch for that.
          I glanced over the patch and it looks pretty good. Nice test coverage!

          This is a pretty big change for 0.94. At the same time it would unlock a bunch of new use cases (I had requests for reverse scan from some of our teams here).

          Do you have this in a production cluster? Can you also make a trunk patch? (Would need a patch for trunk anyway, and it would also allow us to run HadoopQA against it).

          Show
          Lars Hofhansl added a comment - HadoopQA does not work against 0.94. Would need a trunk patch for that. I glanced over the patch and it looks pretty good. Nice test coverage! This is a pretty big change for 0.94. At the same time it would unlock a bunch of new use cases (I had requests for reverse scan from some of our teams here). Do you have this in a production cluster? Can you also make a trunk patch? (Would need a patch for trunk anyway, and it would also allow us to run HadoopQA against it).
          Hide
          Liang Xie added a comment -

          Lars Hofhansl, yes, we impletemented reverse scan to fit those business/production scenario at least:
          1) pagination. our mobile IM client(mitalk) has a feature to read historical messages from latest to before.
          2) select min&max(xxx) from rows
          we had enabled it in our production cluster based on 0.94.3 modified.
          Ok, let me try to make a trunk patch

          Show
          Liang Xie added a comment - Lars Hofhansl , yes, we impletemented reverse scan to fit those business/production scenario at least: 1) pagination. our mobile IM client(mitalk) has a feature to read historical messages from latest to before. 2) select min&max(xxx) from rows we had enabled it in our production cluster based on 0.94.3 modified. Ok, let me try to make a trunk patch
          Hide
          chunhui shen added a comment -

          Liang Xie

          Patch looks nice, after a quick view :
          1. MVCC is not checked in reversed-related seek
          2. JoinedHeap is ignored in RegionScanner for reversed scan

          With a reference of current patch, I make a trunk patch, I think we could take a cooperant work on this feature.

          Show
          chunhui shen added a comment - Liang Xie Patch looks nice, after a quick view : 1. MVCC is not checked in reversed-related seek 2. JoinedHeap is ignored in RegionScanner for reversed scan With a reference of current patch, I make a trunk patch, I think we could take a cooperant work on this feature.
          Show
          chunhui shen added a comment - https://reviews.apache.org/r/11371/
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12584673/hbase-4811-trunkv1.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 15 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5815//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5815//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5815//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5815//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5815//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5815//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5815//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5815//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5815//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5815//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12584673/hbase-4811-trunkv1.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 15 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. -1 javadoc . The javadoc tool appears to have generated 1 warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5815//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5815//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5815//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5815//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5815//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5815//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5815//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5815//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5815//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5815//console This message is automatically generated.
          Hide
          Liang Xie added a comment -

          chunhui shen, thanks a lot

          ps: MVCC is checked by original seek(), which also be invoked in reverse scan path, so here is safe

          Show
          Liang Xie added a comment - chunhui shen , thanks a lot ps: MVCC is checked by original seek(), which also be invoked in reverse scan path, so here is safe
          Hide
          Liang Xie added a comment -

          yes, we need to take a look at JoinedHeap more carefully probably, which is absent in 0.94.3......

          Show
          Liang Xie added a comment - yes, we need to take a look at JoinedHeap more carefully probably, which is absent in 0.94.3......
          Hide
          chunhui shen added a comment -

          ps: MVCC is checked by original seek(), which also be invoked in reverse scan path, so here is safe

          Yes, it's invoked, I see. What I'm worried is that the conflict between seekBeforeRow and skipKVs by MVCC.

          e.g Suppose the following keyvalues:
          1.row1/c1:q1/
          2.row2/c1:q1/
          3.row3/c1:q1/

          If the current peek of storeFileScanner is row2/c1:q1/, then call seekBeforeRow("row2"), the scanner should be at "row1".
          However, if 'row1' cann't be seen because of MVCC, skipKVs will cause scanner at the position of "row2".

          I'm not sure whether you have considered the above case, if so, that's great!

          Show
          chunhui shen added a comment - ps: MVCC is checked by original seek(), which also be invoked in reverse scan path, so here is safe Yes, it's invoked, I see. What I'm worried is that the conflict between seekBeforeRow and skipKVs by MVCC. e.g Suppose the following keyvalues: 1.row1/c1:q1/ 2.row2/c1:q1/ 3.row3/c1:q1/ If the current peek of storeFileScanner is row2/c1:q1/, then call seekBeforeRow("row2"), the scanner should be at "row1". However, if 'row1' cann't be seen because of MVCC, skipKVs will cause scanner at the position of "row2". I'm not sure whether you have considered the above case, if so, that's great!
          Hide
          Liang Xie added a comment -

          chunhui shen, we'd considered your above case before, while trying to figure out the modified TestAcidGuarantees case failure
          thanks

          Show
          Liang Xie added a comment - chunhui shen , we'd considered your above case before, while trying to figure out the modified TestAcidGuarantees case failure thanks
          Hide
          chunhui shen added a comment -

          Addressing Ted's review comments

          Show
          chunhui shen added a comment - Addressing Ted's review comments
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12585901/4811-trunk-v5.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 15 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          -1 javadoc. The javadoc tool appears to have generated 2 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5911//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5911//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12585901/4811-trunk-v5.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 15 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. -1 javadoc . The javadoc tool appears to have generated 2 warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5911//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5911//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5911//console This message is automatically generated.
          Hide
          chunhui shen added a comment -

          fix test case and javadoc warnings

          Show
          chunhui shen added a comment - fix test case and javadoc warnings
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12586037/hbase-4811-trunkv6.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 18 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5924//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5924//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5924//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5924//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5924//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5924//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5924//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5924//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5924//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5924//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12586037/hbase-4811-trunkv6.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 18 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5924//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5924//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5924//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5924//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5924//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5924//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5924//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5924//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5924//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5924//console This message is automatically generated.
          Hide
          Lars Hofhansl added a comment -

          Any chance to put this up on RB for a final check before commit?

          I would also like solicit opinions w.r.t. on 0.94. I am interested in getting this into 0.94 for some Salesforce usecases.

          Show
          Lars Hofhansl added a comment - Any chance to put this up on RB for a final check before commit? I would also like solicit opinions w.r.t. on 0.94. I am interested in getting this into 0.94 for some Salesforce usecases.
          Hide
          Ted Yu added a comment -

          @Lars:
          You can find latest patch here:
          https://reviews.apache.org/r/11371/diff/#index_header

          Show
          Ted Yu added a comment - @Lars: You can find latest patch here: https://reviews.apache.org/r/11371/diff/#index_header
          Hide
          Lars Hofhansl added a comment -

          Added a bunch of comments to RB.
          Generally I think the implementation leaks too many details up the scanner stack. Instead of exposing the special backwardSeek, etc, methods everywhere we should be able to set up the scanner tree with reverse scanners, that just seek/next in the right direction. Only StoreFileScanner and MemstoreScanner need to the special it seems.

          Show
          Lars Hofhansl added a comment - Added a bunch of comments to RB. Generally I think the implementation leaks too many details up the scanner stack. Instead of exposing the special backwardSeek, etc, methods everywhere we should be able to set up the scanner tree with reverse scanners, that just seek/next in the right direction. Only StoreFileScanner and MemstoreScanner need to the special it seems.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12586252/hbase-4811-trunkv7.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 18 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5943//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5943//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5943//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5943//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5943//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5943//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5943//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5943//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5943//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5943//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12586252/hbase-4811-trunkv7.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 18 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5943//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5943//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5943//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5943//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5943//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5943//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5943//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5943//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5943//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5943//console This message is automatically generated.
          Hide
          Lars Hofhansl added a comment -

          I think in its current state this patch is not committable.

          Show
          Lars Hofhansl added a comment - I think in its current state this patch is not committable.
          Hide
          chunhui shen added a comment -

          Lars Hofhansl
          I have replied your comments on review board.

          For current implementation, I think it could be easy to distinguish with general forward scan from understanding view.

          Also it would less touch the code of current core class.

          Show
          chunhui shen added a comment - Lars Hofhansl I have replied your comments on review board. For current implementation, I think it could be easy to distinguish with general forward scan from understanding view. Also it would less touch the code of current core class.
          Hide
          Lars Hofhansl added a comment -

          The idea is great, I love that feature. It was long deemed impossible to do this at all.
          But the patch can be improved. Can we at least get rid of the class casts?

          -0 from me for the current form. (I'm not going to block it)

          Maybe somebody else should have a look too (Stack, Sergey Shelukhin, ramkrishna vasudevan, Ted Yu)?

          I will have some time after HBaseCon, so I can have a try at improving the patch (and as I said the general architecture is sound).

          Show
          Lars Hofhansl added a comment - The idea is great, I love that feature. It was long deemed impossible to do this at all. But the patch can be improved. Can we at least get rid of the class casts? -0 from me for the current form. (I'm not going to block it) Maybe somebody else should have a look too ( Stack , Sergey Shelukhin , ramkrishna vasudevan , Ted Yu )? I will have some time after HBaseCon, so I can have a try at improving the patch (and as I said the general architecture is sound).
          Hide
          chunhui shen added a comment -

          Patch v8 getting rid of class casts and renaming classes

          Lars Hofhansl
          Take a look again?

          Show
          chunhui shen added a comment - Patch v8 getting rid of class casts and renaming classes Lars Hofhansl Take a look again?
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12586924/hbase-4811-trunkv9.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 18 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5981//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5981//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12586924/hbase-4811-trunkv9.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 18 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5981//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5981//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5981//console This message is automatically generated.
          Hide
          Lars Hofhansl added a comment -

          Thank you Chunhui! Was busy today, will have a look tomorrow.

          Show
          Lars Hofhansl added a comment - Thank you Chunhui! Was busy today, will have a look tomorrow.
          Hide
          Lars Hofhansl added a comment -

          v9/v10 is much nicer. Few comments:

          • do we need NonReversedNonLazyKeyValueScanner? Could add "unsupported" implementations for these methods to NonLazyKeyValueScanner.
          • Instead of leaking backwardSeek and seekToLastRow out of the Reversed* classes, should we have an initScan() (or maybe setup()) method on the scanners that does the right thing? I.e. a ReversedScanner would do the seekToLastRow/backwardSeek stuff, and a normal scanner would just seek.
          • This:
            +  @Override
            +  public synchronized boolean reseek(KeyValue kv) throws IOException {
            +    checkReseek();
            +    return heap.backwardSeek(kv);
            +  }
            

            and this

            +  @Override
            +  public boolean backwardSeek(KeyValue key) throws IOException {
            +    checkReseek();
            +    return this.heap.backwardSeek(key);
            +  }
            

            Is weird. It should either scan backwards or not? If we do what I suggested in the previous point, we would not need this, I think.

          That way only MemstoreScanner and StoreFileScanner would be special. And they have to special, because they are opened ahead of time (well, at least StoreFileScanner is).

          Sorry for being pain in the ***.

          Show
          Lars Hofhansl added a comment - v9/v10 is much nicer. Few comments: do we need NonReversedNonLazyKeyValueScanner? Could add "unsupported" implementations for these methods to NonLazyKeyValueScanner. Instead of leaking backwardSeek and seekToLastRow out of the Reversed* classes, should we have an initScan() (or maybe setup()) method on the scanners that does the right thing? I.e. a ReversedScanner would do the seekToLastRow/backwardSeek stuff, and a normal scanner would just seek. This: + @Override + public synchronized boolean reseek(KeyValue kv) throws IOException { + checkReseek(); + return heap.backwardSeek(kv); + } and this + @Override + public boolean backwardSeek(KeyValue key) throws IOException { + checkReseek(); + return this .heap.backwardSeek(key); + } Is weird. It should either scan backwards or not? If we do what I suggested in the previous point, we would not need this, I think. That way only MemstoreScanner and StoreFileScanner would be special. And they have to special, because they are opened ahead of time (well, at least StoreFileScanner is). Sorry for being pain in the ***.
          Hide
          ramkrishna.s.vasudevan added a comment -

          will take a look at this tomorrow.

          Show
          ramkrishna.s.vasudevan added a comment - will take a look at this tomorrow.
          Hide
          chunhui shen added a comment -

          Lars Hofhansl
          Thanks for the review.

          do we need NonReversedNonLazyKeyValueScanner? Could add "unsupported" implementations for these methods to NonLazyKeyValueScanner.

          Sure, we could. Should we need rename "NonLazyKeyValueScanner"? Or only add new annotate for this class?

          should we have an initScan() (or maybe setup()) method on the scanners that does the right thing? I.e. a ReversedScanner would do the seekToLastRow/backwardSeek stuff, and a normal scanner would just seek.

          backwardSeek is not only called when setting up. For this point, I don't understand very much, it would be better if you could update the patch as you think.

          It should either scan backwards or not?

          It is indeed weird calling backwardSeek in the method of reseek since reseek means forward seek.

          I would update the path with addressing the first and third point later

          Show
          chunhui shen added a comment - Lars Hofhansl Thanks for the review. do we need NonReversedNonLazyKeyValueScanner? Could add "unsupported" implementations for these methods to NonLazyKeyValueScanner. Sure, we could. Should we need rename "NonLazyKeyValueScanner"? Or only add new annotate for this class? should we have an initScan() (or maybe setup()) method on the scanners that does the right thing? I.e. a ReversedScanner would do the seekToLastRow/backwardSeek stuff, and a normal scanner would just seek. backwardSeek is not only called when setting up. For this point, I don't understand very much, it would be better if you could update the patch as you think. It should either scan backwards or not? It is indeed weird calling backwardSeek in the method of reseek since reseek means forward seek. I would update the path with addressing the first and third point later
          Hide
          Lars Hofhansl added a comment -

          Sure, we could. Should we need rename "NonLazyKeyValueScanner"? Or only add new annotate for this class?

          I'd just add the new signatures.

          backwardSeek is not only called when setting up

          I was trying to say that:

          1. ReversedKeyValueHeap should not need backwardSeek, since all calls to (re)seek, etc, call backwardSeek anyway.
          2. Only MemstoreScanner and StoreFileScanner should need an explicit backwardSeek method. Is that not so? (as I said I probably missed something).

          For a reversed scan we should get the following hierarchy:
          ReversedRegionScanner -> ReversedKVHeap -> ReversedStoreScanner -> ReversedKVHeap -> (MemstoreScanner|StoreFileScanner). Right?

          ReversedStoreScanner could always do the right (backward) seeking when (re)seek, etc, is called. The tricky parts of MemstoreScanner and StoreFileScanner since they are create/cached per memstore/HFile.

          So would: ReversedRegionScanner -> KVHelp -> RerversedStoreScanner -> ReversedKVHeap -> (MemstoreScanner|StoreFileScanner) work?
          The ReversedKVHeap would only be needed to deal with the specialness of MemstoreScanner and StoreFileScanner.

          Show
          Lars Hofhansl added a comment - Sure, we could. Should we need rename "NonLazyKeyValueScanner"? Or only add new annotate for this class? I'd just add the new signatures. backwardSeek is not only called when setting up I was trying to say that: ReversedKeyValueHeap should not need backwardSeek, since all calls to (re)seek, etc, call backwardSeek anyway. Only MemstoreScanner and StoreFileScanner should need an explicit backwardSeek method. Is that not so? (as I said I probably missed something). For a reversed scan we should get the following hierarchy: ReversedRegionScanner -> ReversedKVHeap -> ReversedStoreScanner -> ReversedKVHeap -> (MemstoreScanner|StoreFileScanner). Right? ReversedStoreScanner could always do the right (backward) seeking when (re)seek, etc, is called. The tricky parts of MemstoreScanner and StoreFileScanner since they are create/cached per memstore/HFile. So would: ReversedRegionScanner -> KVHelp -> RerversedStoreScanner -> ReversedKVHeap -> (MemstoreScanner|StoreFileScanner) work? The ReversedKVHeap would only be needed to deal with the specialness of MemstoreScanner and StoreFileScanner.
          Hide
          chunhui shen added a comment -

          So would: ReversedRegionScanner -> KVHelp -> RerversedStoreScanner -> ReversedKVHeap -> (MemstoreScanner|StoreFileScanner) work?

          Need ReversedRegionScanner -> ReversedKVHeap, because we need a reversed sort of scanners in ReversedKVHeap and call seekToPreviousRow for RegionScanner#nextRow, otherwise the scan order would be wrong.

          ReversedKeyValueHeap should not need backwardSeek, since all calls to (re)seek, etc, call backwardSeek anyway.

          ReversedKeyValueHeap should support backwardSeek and seekToPreviousRow even if no calls. Is it better to understand that prohibit calling (re)seek in ReversedKeyValueHeap and ReversedStoreScanner

          do we need NonReversedNonLazyKeyValueScanner? Could add "unsupported" implementations for these methods to NonLazyKeyValueScanner.

          MemStoreScanner extends from NonLazyKeyValueScanner, it seems not appropriate if add "unsupported" implementations to NonLazyKeyValueScanner

          Show
          chunhui shen added a comment - So would: ReversedRegionScanner -> KVHelp -> RerversedStoreScanner -> ReversedKVHeap -> (MemstoreScanner|StoreFileScanner) work? Need ReversedRegionScanner -> ReversedKVHeap, because we need a reversed sort of scanners in ReversedKVHeap and call seekToPreviousRow for RegionScanner#nextRow, otherwise the scan order would be wrong. ReversedKeyValueHeap should not need backwardSeek, since all calls to (re)seek, etc, call backwardSeek anyway. ReversedKeyValueHeap should support backwardSeek and seekToPreviousRow even if no calls. Is it better to understand that prohibit calling (re)seek in ReversedKeyValueHeap and ReversedStoreScanner do we need NonReversedNonLazyKeyValueScanner? Could add "unsupported" implementations for these methods to NonLazyKeyValueScanner. MemStoreScanner extends from NonLazyKeyValueScanner, it seems not appropriate if add "unsupported" implementations to NonLazyKeyValueScanner
          Hide
          chunhui shen added a comment -

          Addressing review comments and adjust class/method structure a little

          Show
          chunhui shen added a comment - Addressing review comments and adjust class/method structure a little
          Hide
          Lars Hofhansl added a comment -

          stack, ramkrishna.s.vasudevan, Nicolas Liochon, could you guys have a look as well, please?

          Show
          Lars Hofhansl added a comment - stack , ramkrishna.s.vasudevan , Nicolas Liochon , could you guys have a look as well, please?
          Hide
          Liang Xie added a comment -

          chunhui shen, we need to change one line in HFileReaderV2.seekBefore() in trunk patch(but not 0.94's):

                if (reader.getComparator().compare(firstKey.array(),
                    firstKey.arrayOffset(), firstKey.limit(), key, offset, length) == 0)
          

          to:

                if (reader.getComparator().compare(firstKey.array(),
                    firstKey.arrayOffset(), firstKey.limit(), key, offset, length) >= 0)
          

          Because HBASE-7845(optimize hfile index key) was in trunk already, it'll have an impact here. We(Xiaomi) are running a 0.94 modified version with HBASE-7845 in production cluster, and changed "==" to ">=".

          Show
          Liang Xie added a comment - chunhui shen , we need to change one line in HFileReaderV2.seekBefore() in trunk patch(but not 0.94's): if (reader.getComparator().compare(firstKey.array(), firstKey.arrayOffset(), firstKey.limit(), key, offset, length) == 0) to: if (reader.getComparator().compare(firstKey.array(), firstKey.arrayOffset(), firstKey.limit(), key, offset, length) >= 0) Because HBASE-7845 (optimize hfile index key) was in trunk already, it'll have an impact here. We(Xiaomi) are running a 0.94 modified version with HBASE-7845 in production cluster, and changed "==" to ">=".
          Hide
          Liang Xie added a comment -

          btw, my attached original 0.94-v2.txt seems didn't enable *lazy seek" for reverse scan, and we had supported it in our internal codebase, i can make a new 0.94 patch later.

          Show
          Liang Xie added a comment - btw, my attached original 0.94-v2.txt seems didn't enable *lazy seek" for reverse scan, and we had supported it in our internal codebase, i can make a new 0.94 patch later.
          Hide
          chunhui shen added a comment -

          we need to change one line in HFileReaderV2.seekBefore() in trunk patch(but not 0.94's):

          I have found and changed this one in current patch.

          didn't enable *lazy seek" for reverse scan, and we had supported it in our internal codebase, i can make a new 0.94 patch later.

          It's nice, we could import it in a new improvement issue after this feature is stable.

          Recently I have found one bug in ClientScanner if the reversed scan cross two or more regions(The scan order will be wrong between regions in ClientScanner). I will upload the new patch with fixing this bug later.

          Show
          chunhui shen added a comment - we need to change one line in HFileReaderV2.seekBefore() in trunk patch(but not 0.94's): I have found and changed this one in current patch. didn't enable *lazy seek" for reverse scan, and we had supported it in our internal codebase, i can make a new 0.94 patch later. It's nice, we could import it in a new improvement issue after this feature is stable. Recently I have found one bug in ClientScanner if the reversed scan cross two or more regions(The scan order will be wrong between regions in ClientScanner). I will upload the new patch with fixing this bug later.
          Hide
          ramkrishna.s.vasudevan added a comment -

          @LArs
          Sorry could not find some time on this. Anyway Chunhui has said he would update the patch. Let me take a look after that.

          Show
          ramkrishna.s.vasudevan added a comment - @LArs Sorry could not find some time on this. Anyway Chunhui has said he would update the patch. Let me take a look after that.
          Hide
          Nicolas Liochon added a comment -

          It's in my todo list as well. As Ram, I will review as soon as Chunhui updates it.

          Show
          Nicolas Liochon added a comment - It's in my todo list as well. As Ram, I will review as soon as Chunhui updates it.
          Hide
          chunhui shen added a comment -

          Patch-V12 fix one bug in ClientScanner for reversed scan

          rb: https://reviews.apache.org/r/11371/

          Show
          chunhui shen added a comment - Patch-V12 fix one bug in ClientScanner for reversed scan rb: https://reviews.apache.org/r/11371/
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12590562/hbase-4811-trunkv12.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 18 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          -1 javadoc. The javadoc tool appears to have generated 2 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor
          org.apache.hadoop.hbase.client.TestFromClientSide

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6199//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6199//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6199//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6199//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6199//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6199//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6199//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6199//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6199//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6199//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12590562/hbase-4811-trunkv12.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 18 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. -1 javadoc . The javadoc tool appears to have generated 2 warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor org.apache.hadoop.hbase.client.TestFromClientSide Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6199//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6199//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6199//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6199//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6199//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6199//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6199//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6199//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6199//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6199//console This message is automatically generated.
          Hide
          chunhui shen added a comment -

          Fixing failed tests

          Show
          chunhui shen added a comment - Fixing failed tests
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12590578/hbase-4811-trunkv13.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 18 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          -1 javadoc. The javadoc tool appears to have generated 2 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6200//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6200//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6200//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6200//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6200//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6200//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6200//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6200//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6200//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6200//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12590578/hbase-4811-trunkv13.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 18 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. -1 javadoc . The javadoc tool appears to have generated 2 warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6200//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6200//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6200//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6200//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6200//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6200//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6200//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6200//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6200//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6200//console This message is automatically generated.
          Hide
          chunhui shen added a comment -

          Addressing ram's rb comments

          Show
          chunhui shen added a comment - Addressing ram's rb comments
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12591365/hbase-4811-trunkv14.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 18 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          -1 javadoc. The javadoc tool appears to have generated 2 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6262//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6262//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12591365/hbase-4811-trunkv14.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 18 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. -1 javadoc . The javadoc tool appears to have generated 2 warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6262//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6262//console This message is automatically generated.
          Hide
          Lars Hofhansl added a comment -

          I think we dropped the ball on this. We should finish this. How do other folks feel about this.
          The feature is great! We just have to be careful to avoid adding unnecessary APIs to the core scanner classes.

          Show
          Lars Hofhansl added a comment - I think we dropped the ball on this. We should finish this. How do other folks feel about this. The feature is great! We just have to be careful to avoid adding unnecessary APIs to the core scanner classes.
          Hide
          chunhui shen added a comment -

          Upload the newest patch based on trunk

          Show
          chunhui shen added a comment - Upload the newest patch based on trunk
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12598371/hbase-4811-trunkv16.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 18 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6779//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6779//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6779//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6779//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6779//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6779//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6779//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6779//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6779//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6779//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12598371/hbase-4811-trunkv16.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 18 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6779//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6779//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6779//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6779//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6779//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6779//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6779//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6779//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6779//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6779//console This message is automatically generated.
          Hide
          stack added a comment -

          I got a strong vote from a customer of the place I work at.

          Dan Burkett out on mailing list asked for this too.

          Any chance of reviews?

          Show
          stack added a comment - I got a strong vote from a customer of the place I work at. Dan Burkett out on mailing list asked for this too. Any chance of reviews?
          Hide
          Lars Hofhansl added a comment -

          I am +100 on the feature (heck, I was convinced that it cannot be done). +0 on the patch, though.
          If I am the only one who is worried about the extra reverseXXX methods all over the scanner classes then let's commit.

          Show
          Lars Hofhansl added a comment - I am +100 on the feature (heck, I was convinced that it cannot be done). +0 on the patch, though. If I am the only one who is worried about the extra reverseXXX methods all over the scanner classes then let's commit.
          Hide
          Lars Hofhansl added a comment -

          And honestly I want this in 0.94, but that's a different story.

          Show
          Lars Hofhansl added a comment - And honestly I want this in 0.94, but that's a different story.
          Hide
          Lars Hofhansl added a comment -

          Actually looking at HBase-4811-0.94-v2 again. All the backwardSeek stuff that I did not like is gone (maybe I was looking at an older version?)
          So I'll change my vote for the 0.94 patch to +1. It's a new feature that does not impact current behavior. The only part in 0.94 that needs to be changed is we use Scan attributes to pass the reverse flag in a backward compatible way, and that is an easy change.

          (And we also want this for Phoenix, so that we can support DESC scans without duplicating the data)

          Show
          Lars Hofhansl added a comment - Actually looking at HBase-4811-0.94-v2 again. All the backwardSeek stuff that I did not like is gone (maybe I was looking at an older version?) So I'll change my vote for the 0.94 patch to +1. It's a new feature that does not impact current behavior. The only part in 0.94 that needs to be changed is we use Scan attributes to pass the reverse flag in a backward compatible way, and that is an easy change. (And we also want this for Phoenix, so that we can support DESC scans without duplicating the data)
          Hide
          Lars Hofhansl added a comment -

          chunhui shen, is the 0.94 up-to-date with the trunk patch you attached?

          Show
          Lars Hofhansl added a comment - chunhui shen , is the 0.94 up-to-date with the trunk patch you attached?
          Hide
          Lars Hofhansl added a comment -

          Looks like this:

          +        cur = hfs.getKeyValue();
          +        //return skipKVsNewerThanReadpoint();
          +        //CAUTION!!! see http://phabricator.n.miliao.com/T293 for more detail
          +        //in short, the original intention of seekBefore() is moving to a smaller kv, but
          +        //skipKVsNewerThanReadpoint could break this assumption under special cases.
          +        return true;
          

          Is fixed in the trunk patch, but not in 0.94.

          Show
          Lars Hofhansl added a comment - Looks like this: + cur = hfs.getKeyValue(); + // return skipKVsNewerThanReadpoint(); + //CAUTION!!! see http://phabricator.n.miliao.com/T293 for more detail + //in short , the original intention of seekBefore() is moving to a smaller kv, but + //skipKVsNewerThanReadpoint could break this assumption under special cases. + return true ; Is fixed in the trunk patch, but not in 0.94.
          Hide
          Lars Hofhansl added a comment -

          Version that handles used Scan attributes to set/restore the reverse flag.
          Still need to fix the MVCC stuff for backward scans like done in trunk.

          Show
          Lars Hofhansl added a comment - Version that handles used Scan attributes to set/restore the reverse flag. Still need to fix the MVCC stuff for backward scans like done in trunk.
          Hide
          Lars Hofhansl added a comment -

          Hmm... The trunk version still has the backward seek stuff in there. I'm somewhat confused now.
          The 0.94 version is quite a bit older than the trunk patch.

          Show
          Lars Hofhansl added a comment - Hmm... The trunk version still has the backward seek stuff in there. I'm somewhat confused now. The 0.94 version is quite a bit older than the trunk patch.
          Hide
          Lars Hofhansl added a comment -

          So the backwardSeek is needed to support lazy seek.
          If other's are fine with this, I won't block it. Patch locks correct.

          chunhui shen Any chance for a 0.94 patch with the lazy seek stuff in it (and using the Scan attributes code that I illustrated)?

          Show
          Lars Hofhansl added a comment - So the backwardSeek is needed to support lazy seek. If other's are fine with this, I won't block it. Patch locks correct. chunhui shen Any chance for a 0.94 patch with the lazy seek stuff in it (and using the Scan attributes code that I illustrated)?
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12599757/4811-0.94-v3.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 19 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6868//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12599757/4811-0.94-v3.txt against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 19 new or modified tests. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6868//console This message is automatically generated.
          Hide
          chunhui shen added a comment -

          Lars Hofhansl
          The patch for 0.94 version is quite old, should be re-made.

          Lazy seek is able to be applied in reversed scan.
          And have to change some methods from doing real seek to request seek.

          If necessary, I could make the new patch with lazy seek, else I prefer to take it as a new improvement issue

          Show
          chunhui shen added a comment - Lars Hofhansl The patch for 0.94 version is quite old, should be re-made. Lazy seek is able to be applied in reversed scan. And have to change some methods from doing real seek to request seek. If necessary, I could make the new patch with lazy seek, else I prefer to take it as a new improvement issue
          Hide
          chunhui shen added a comment -

          regenerating patch-v17 with protoc 2.5

          Show
          chunhui shen added a comment - regenerating patch-v17 with protoc 2.5
          Hide
          Lars Hofhansl added a comment -

          Does the trunk patch do lazy seek? If not, let's do it in another issue as you suggest.

          Show
          Lars Hofhansl added a comment - Does the trunk patch do lazy seek? If not, let's do it in another issue as you suggest.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12599866/hbase-4811-trunkv17.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 18 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

          -1 release audit. The applied patch generated 2 release audit warnings (more than the trunk's current 0 warnings).

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12599866/hbase-4811-trunkv17.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 18 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. -1 findbugs . The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. -1 release audit . The applied patch generated 2 release audit warnings (more than the trunk's current 0 warnings). +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6894//console This message is automatically generated.
          Hide
          chunhui shen added a comment -

          Does the trunk patch do lazy seek?

          No.

          Show
          chunhui shen added a comment - Does the trunk patch do lazy seek? No.
          Hide
          Ted Yu added a comment -

          In tests:

          +    InternalScanner scanner = this.region.getScanner(s);
          +    while (true) {
          +      ArrayList<KeyValue> values = new ArrayList<KeyValue>();
          +      boolean isMoreResults = scanner.next(values);
          

          Please close the scanner at the end of the test.

          +  public void testReverseScanner_FromMemStoreAndHFiles_MultiCFs1()
          

          Can the above test be given more specific name (same for testReverseScanner_FromMemStoreAndHFiles_MultiCFs2) ?

          +   * if they have the equal row, sort smaller one before than bigger one
          

          typo: 'than' -> 'the'

          In NonReversedNonLazyKeyValueScanner:

          +  public boolean seekToPreviousRow(KeyValue key) throws IOException {
          +    throw new NotImplementedException("seekToPreviousRow must not be called on a "
          

          But from ReversedKeyValueHeap:

          +  public boolean seekToLastRow() throws IOException {
          +    throw new UnsupportedOperationException("Not implemented");
          

          It would be nice to use the same exception for consistency.

          Show
          Ted Yu added a comment - In tests: + InternalScanner scanner = this .region.getScanner(s); + while ( true ) { + ArrayList<KeyValue> values = new ArrayList<KeyValue>(); + boolean isMoreResults = scanner.next(values); Please close the scanner at the end of the test. + public void testReverseScanner_FromMemStoreAndHFiles_MultiCFs1() Can the above test be given more specific name (same for testReverseScanner_FromMemStoreAndHFiles_MultiCFs2) ? + * if they have the equal row, sort smaller one before than bigger one typo: 'than' -> 'the' In NonReversedNonLazyKeyValueScanner: + public boolean seekToPreviousRow(KeyValue key) throws IOException { + throw new NotImplementedException( "seekToPreviousRow must not be called on a " But from ReversedKeyValueHeap: + public boolean seekToLastRow() throws IOException { + throw new UnsupportedOperationException( "Not implemented" ); It would be nice to use the same exception for consistency.
          Hide
          chunhui shen added a comment -

          Addressing the above Ted's comments and upload it on review board also.

          https://reviews.apache.org/r/11371/

          Show
          chunhui shen added a comment - Addressing the above Ted's comments and upload it on review board also. https://reviews.apache.org/r/11371/
          Hide
          Andrew Purtell added a comment -

          There was some discussion on including this in 0.98 on the dev@ list. Seems in a reasonable state already, but the changes will need to be rebased after HBASE-9245 and subtasks.

          Show
          Andrew Purtell added a comment - There was some discussion on including this in 0.98 on the dev@ list. Seems in a reasonable state already, but the changes will need to be rebased after HBASE-9245 and subtasks.
          Hide
          Lars Hofhansl added a comment -

          chunhui shen

          The patch for 0.94 version is quite old, should be re-made.

          You mean it's need to be rebased (it still applies mostly fine), or that you added a lot of stuff to the trunk version that should be in the 0.94 version as well?

          Show
          Lars Hofhansl added a comment - chunhui shen The patch for 0.94 version is quite old, should be re-made. You mean it's need to be rebased (it still applies mostly fine), or that you added a lot of stuff to the trunk version that should be in the 0.94 version as well?
          Hide
          chunhui shen added a comment -

          you added a lot of stuff to the trunk version that should be in the 0.94 version as well?

          Trunk patch has more tests and fix some additional bugs.

          Once the trunk patch passed, I would make the corresponding 0.94 version patch

          Show
          chunhui shen added a comment - you added a lot of stuff to the trunk version that should be in the 0.94 version as well? Trunk patch has more tests and fix some additional bugs. Once the trunk patch passed, I would make the corresponding 0.94 version patch
          Hide
          chunhui shen added a comment -

          Addressing Ted's comments on rb

          Show
          chunhui shen added a comment - Addressing Ted's comments on rb
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12600317/hbase-4811-trunkv19.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 18 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6942//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6942//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6942//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6942//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6942//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6942//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6942//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6942//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6942//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6942//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12600317/hbase-4811-trunkv19.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 18 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6942//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6942//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6942//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6942//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6942//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6942//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6942//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6942//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6942//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6942//console This message is automatically generated.
          Hide
          Andrew Purtell added a comment -

          There was some discussion about this on dev@ recently. To summarize:

          • Why do we need backwardSeek() if we also have reverse scanners and heaps?
          • Is it reasonable if we say users should not care about 'hbase.client.reversedscanner.maxbyte.length'? Can this be done another way?
          Show
          Andrew Purtell added a comment - There was some discussion about this on dev@ recently. To summarize: Why do we need backwardSeek() if we also have reverse scanners and heaps? Is it reasonable if we say users should not care about 'hbase.client.reversedscanner.maxbyte.length'? Can this be done another way?
          Hide
          chunhui shen added a comment - - edited

          Why do we need backwardSeek() if we also have reverse scanners and heaps?

          Reverse scanners' reversed scaning is based on the backwardSeek(), backwardSeek() is a seek like reseek(), but it won't seek to the rows after seekKey.(See the detail javadoc of KeyValueScanner#backwardSeek from the patch)

          Is it reasonable if we say users should not care about 'hbase.client.reversedscanner.maxbyte.length'?

          I will try to remove this flaw in next days

          Show
          chunhui shen added a comment - - edited Why do we need backwardSeek() if we also have reverse scanners and heaps? Reverse scanners' reversed scaning is based on the backwardSeek(), backwardSeek() is a seek like reseek(), but it won't seek to the rows after seekKey.(See the detail javadoc of KeyValueScanner#backwardSeek from the patch) Is it reasonable if we say users should not care about 'hbase.client.reversedscanner.maxbyte.length'? I will try to remove this flaw in next days
          Hide
          chunhui shen added a comment -

          Patch-v20 removes the config in Reversed Scan and now it's no necessary to set start row in Reversed Scan

          Show
          chunhui shen added a comment - Patch-v20 removes the config in Reversed Scan and now it's no necessary to set start row in Reversed Scan
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12601109/hbase-4811-trunkv20.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 18 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/7006//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7006//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7006//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7006//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7006//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7006//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7006//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7006//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7006//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/7006//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12601109/hbase-4811-trunkv20.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 18 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/7006//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7006//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7006//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7006//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7006//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7006//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7006//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7006//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7006//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/7006//console This message is automatically generated.
          Hide
          chunhui shen added a comment -

          Andrew Purtell
          Lars Hofhansl

          Could you take a look at the latest patch?
          Thanks

          Show
          chunhui shen added a comment - Andrew Purtell Lars Hofhansl Could you take a look at the latest patch? Thanks
          Hide
          Liang Xie added a comment -

          seems no progress ?

          Show
          Liang Xie added a comment - seems no progress ?
          Hide
          Anil Gupta added a comment -

          Hi All,

          I am running HBase0.94.6. We need this feature. Can we backport this to 0.94? If yes, then how hard the backporting would be?

          Thanks,
          Anil

          Show
          Anil Gupta added a comment - Hi All, I am running HBase0.94.6. We need this feature. Can we backport this to 0.94? If yes, then how hard the backporting would be? Thanks, Anil
          Hide
          Andrew Purtell added a comment -

          Lars Hofhansl what do you think about this:

          Why do we need backwardSeek() if we also have reverse scanners and heaps?

          Reverse scanners' reversed scaning is based on the backwardSeek(), backwardSeek() is a seek like reseek(), but it won't seek to the rows after seekKey.(See the detail javadoc of KeyValueScanner#backwardSeek from the patch)

          Show
          Andrew Purtell added a comment - Lars Hofhansl what do you think about this: Why do we need backwardSeek() if we also have reverse scanners and heaps? Reverse scanners' reversed scaning is based on the backwardSeek(), backwardSeek() is a seek like reseek(), but it won't seek to the rows after seekKey.(See the detail javadoc of KeyValueScanner#backwardSeek from the patch)
          Hide
          Lars Hofhansl added a comment -

          I still don't understand why a reverse scanner's reseek() can't do exactly what backwardSeek() is doing.
          In any case, if I am the only one with issues it would be likely that I am missing something. This is an often requested and important feature, so let's get it in.

          Lemme do some basic performance tests to make sure we didn't regress there.
          chunhui shen, mind making a 0.94 patch with the latest?

          Show
          Lars Hofhansl added a comment - I still don't understand why a reverse scanner's reseek() can't do exactly what backwardSeek() is doing. In any case, if I am the only one with issues it would be likely that I am missing something. This is an often requested and important feature, so let's get it in. Lemme do some basic performance tests to make sure we didn't regress there. chunhui shen , mind making a 0.94 patch with the latest?
          Hide
          chunhui shen added a comment -

          Rebasing the trunk patch and

          making the 0.94-branch patch with the latest

          Show
          chunhui shen added a comment - Rebasing the trunk patch and making the 0.94-branch patch with the latest
          Hide
          Lars Hofhansl added a comment -

          Thanks chunhui shen. Will run some perf tests on normal scans. If there're no new performance issues, let's commit to 0.94+.

          Andrew Purtell, Ted Yu, stack, agreed?

          Show
          Lars Hofhansl added a comment - Thanks chunhui shen . Will run some perf tests on normal scans. If there're no new performance issues, let's commit to 0.94+. Andrew Purtell , Ted Yu , stack , agreed?
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12605768/hbase-4811-trunkv21.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 18 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12605768/hbase-4811-trunkv21.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 18 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/7410//console This message is automatically generated.
          Hide
          Andrew Purtell added a comment -

          Thanks chunhui shen for removing the need for 'hbase.client.reversedscanner.maxbyte.length'.

          Will run some perf tests on normal scans. If there're no new performance issues, let's commit to 0.94+.

          Lars Hofhansl: +1, it's been requested a few times, let's try it out.

          Show
          Andrew Purtell added a comment - Thanks chunhui shen for removing the need for 'hbase.client.reversedscanner.maxbyte.length'. Will run some perf tests on normal scans. If there're no new performance issues, let's commit to 0.94+. Lars Hofhansl : +1, it's been requested a few times, let's try it out.
          Hide
          Lars Hofhansl added a comment -

          So it turns out that this does slow down scanning in a tight loop (20m rows, 1col, 100 byte values - all filtered on the server by a ValueFilter).
          Average of 5 runs with patch: 17.1s, without patch: 16.1s without patch.

          Show
          Lars Hofhansl added a comment - So it turns out that this does slow down scanning in a tight loop (20m rows, 1col, 100 byte values - all filtered on the server by a ValueFilter). Average of 5 runs with patch: 17.1s, without patch: 16.1s without patch.
          Hide
          Andrew Purtell added a comment -

          That's not good. Will check this on trunk when I have some free time. Chunhui, do you have benchmarks/microbenchmarks from when this was developed or from your app?

          Show
          Andrew Purtell added a comment - That's not good. Will check this on trunk when I have some free time. Chunhui, do you have benchmarks/microbenchmarks from when this was developed or from your app?
          Hide
          Lars Hofhansl added a comment -

          I will double check this on another machine as well (this was with the 0.94 patch).
          Note that if the data is actually returned to the client the difference is not noticeable.

          Show
          Lars Hofhansl added a comment - I will double check this on another machine as well (this was with the 0.94 patch). Note that if the data is actually returned to the client the difference is not noticeable.
          Hide
          Lars Hofhansl added a comment -

          On another (slower) machine. 20m rows, 1 col, 10bytes value this time and with JDK7: with patch: 15.82 without patch: 15.70. So the effect is less pronounced here, but it repeatable. The shortest run is always without the patch and longest with the patch as well. Looking for culprits now.

          Show
          Lars Hofhansl added a comment - On another (slower) machine. 20m rows, 1 col, 10bytes value this time and with JDK7: with patch: 15.82 without patch: 15.70. So the effect is less pronounced here, but it repeatable. The shortest run is always without the patch and longest with the patch as well. Looking for culprits now.
          Hide
          Lars Hofhansl added a comment -

          I thought it might be the double synchronized or the fact that checkScan is now method (rather than just an assert). Indeed changing that has a slight impact, but the patch is still slower.

          Show
          Lars Hofhansl added a comment - I thought it might be the double synchronized or the fact that checkScan is now method (rather than just an assert). Indeed changing that has a slight impact, but the patch is still slower.
          Hide
          chunhui shen added a comment -

          Lars Hofhansl
          Thanks for the performance check .

          What's the related code about "the double synchronized "?

          This patch will add a slight CPU cost for scanning,
          maybe this is the cause of slowing down scanning since the performance of test case is mostly depended on the CPU

          Show
          chunhui shen added a comment - Lars Hofhansl Thanks for the performance check . What's the related code about "the double synchronized "? This patch will add a slight CPU cost for scanning, maybe this is the cause of slowing down scanning since the performance of test case is mostly depended on the CPU
          Hide
          Lars Hofhansl added a comment -

          Sorry, should have stated that I was referring seekToNextRow and seekAsDirection as being synchronized, but calling a synchronized method in the forward case (For the record: I do not think that is a problem)

          My test is indeed CPU bound correct. I skimmed over the patch, and it is not immediately where the extra CPU comes from (except for the cases I mentioned). Do you have a specific part of the code where you think it adds CPU cost to forward scanning, chunhui shen?
          (It's fine if reverse scanning is slower, but forward scanning should not be impacted)

          Show
          Lars Hofhansl added a comment - Sorry, should have stated that I was referring seekToNextRow and seekAsDirection as being synchronized, but calling a synchronized method in the forward case (For the record: I do not think that is a problem) My test is indeed CPU bound correct. I skimmed over the patch, and it is not immediately where the extra CPU comes from (except for the cases I mentioned). Do you have a specific part of the code where you think it adds CPU cost to forward scanning, chunhui shen ? (It's fine if reverse scanning is slower, but forward scanning should not be impacted)
          Hide
          Lars Hofhansl added a comment -

          Looking through the patch I don't see why forward scanning would be slower. There's a bit more function dispatching and a bit more "inheritance", but the JVM should automatically do the right thing (inline methods, etc). So I am a bit baffled.
          Did another test with slightly changed code, still brings the runtime from 17.2 to 18.2 secs (test varies slightly from initial test).

          Show
          Lars Hofhansl added a comment - Looking through the patch I don't see why forward scanning would be slower. There's a bit more function dispatching and a bit more "inheritance", but the JVM should automatically do the right thing (inline methods, etc). So I am a bit baffled. Did another test with slightly changed code, still brings the runtime from 17.2 to 18.2 secs (test varies slightly from initial test).
          Hide
          chunhui shen added a comment -

          Maybe caching the variable of 'reversed' in Scan.class will take something.
          Like:

           public boolean isReversed() {
              if (this.reversed == null) {
                byte[] attr = getAttribute(REVERSED_ATTR);
                this.reversed = attr == null ? false : Bytes.toBoolean(attr);
              }
              return this.reversed;
            }
          
          public void setReversed(boolean reversed) {
              setAttribute(REVERSED_ATTR, Bytes.toBytes(reversed));
              this.reversed = reversed;
            }
          

          I have tested the above performance case, the result is near 15.82s vs 15.70s. But haven't got large difference like 17.1s vs 16.1s.

          After caching the variable of 'reversed' , the performance seems to be better, but I'm not very sure since all the tests are not completely stable in my environment and the gap is small

          Show
          chunhui shen added a comment - Maybe caching the variable of 'reversed' in Scan.class will take something. Like: public boolean isReversed() { if ( this .reversed == null ) { byte [] attr = getAttribute(REVERSED_ATTR); this .reversed = attr == null ? false : Bytes.toBoolean(attr); } return this .reversed; } public void setReversed( boolean reversed) { setAttribute(REVERSED_ATTR, Bytes.toBytes(reversed)); this .reversed = reversed; } I have tested the above performance case, the result is near 15.82s vs 15.70s. But haven't got large difference like 17.1s vs 16.1s. After caching the variable of 'reversed' , the performance seems to be better, but I'm not very sure since all the tests are not completely stable in my environment and the gap is small
          Hide
          Lars Hofhansl added a comment -

          That is one thing I looked at. The other was the use on a member for stopSkippingKVsIfNextRow in StoreFileScsanner (rather than passing that in).
          I'll do some more testing today.

          Show
          Lars Hofhansl added a comment - That is one thing I looked at. The other was the use on a member for stopSkippingKVsIfNextRow in StoreFileScsanner (rather than passing that in). I'll do some more testing today.
          Hide
          Lars Hofhansl added a comment -

          Actually with that change in place, I cannot detect a statistically significant slowdown on my home machine. Lemme me try again on my work machine.

          Show
          Lars Hofhansl added a comment - Actually with that change in place, I cannot detect a statistically significant slowdown on my home machine. Lemme me try again on my work machine.
          Hide
          Lars Hofhansl added a comment -

          Andrew Purtell, did you get any chance to do some perf testing on this?
          As far as I am concerned this is good now in terms of performance.

          Show
          Lars Hofhansl added a comment - Andrew Purtell , did you get any chance to do some perf testing on this? As far as I am concerned this is good now in terms of performance.
          Hide
          Andrew Purtell added a comment -

          Lars Hofhansl I'm sadly underwater, if you think it's good +1 from me

          Show
          Andrew Purtell added a comment - Lars Hofhansl I'm sadly underwater, if you think it's good +1 from me
          Hide
          Lars Hofhansl added a comment -

          Jean-Marc Spaggiari, any chance to run this through your routine. Need to make sure this does not regress the forward scanning.

          Show
          Lars Hofhansl added a comment - Jean-Marc Spaggiari , any chance to run this through your routine. Need to make sure this does not regress the forward scanning.
          Hide
          Lars Hofhansl added a comment -

          Rebased 0.94 patch with the perf improvement Chunhui mentioned.

          Show
          Lars Hofhansl added a comment - Rebased 0.94 patch with the perf improvement Chunhui mentioned.
          Hide
          Jean-Marc Spaggiari added a comment -

          Sure I can.

          So I run the last 0.94 branch and 0.94 branch+ this patch and compare. Is that correct?

          Show
          Jean-Marc Spaggiari added a comment - Sure I can. So I run the last 0.94 branch and 0.94 branch+ this patch and compare. Is that correct?
          Hide
          Lars Hofhansl added a comment -

          Yep. If you have time, that's be great (I just would like an independent performance review here ).

          Show
          Lars Hofhansl added a comment - Yep. If you have time, that's be great (I just would like an independent performance review here ).
          Hide
          Jean-Marc Spaggiari added a comment -

          Do you want all the tests?

          Random read/write
          Sequence read/write
          scans.

          Or just the scans? The more tests, the longer

          Show
          Jean-Marc Spaggiari added a comment - Do you want all the tests? Random read/write Sequence read/write scans. Or just the scans? The more tests, the longer
          Hide
          Jean-Marc Spaggiari added a comment -

          started with:
          filterScan
          randomSeekScan
          scanRange10
          scanRange1000

          for both 0.94 branch and 0.94+4811... Will post the results as soon as it's done. Running on SSD drive. I can also run it on a server with SATA if required.

          Show
          Jean-Marc Spaggiari added a comment - started with: filterScan randomSeekScan scanRange10 scanRange1000 for both 0.94 branch and 0.94+4811... Will post the results as soon as it's done. Running on SSD drive. I can also run it on a server with SATA if required.
          Hide
          Lars Hofhansl added a comment -

          Some gets (to measure any increase setup costs for scanning) and some scans (to measure any new scan overhead).

          Show
          Lars Hofhansl added a comment - Some gets (to measure any increase setup costs for scanning) and some scans (to measure any new scan overhead).
          Hide
          Jean-Marc Spaggiari added a comment -

          Values for gets are running and will be posted very shortly.

          Test 0.94.12 patched 0.94.12 Diff
          org.apache.hadoop.hbase.PerformanceEvaluation$FilteredScanTest 10.62 10.55 99.34%
          org.apache.hadoop.hbase.PerformanceEvaluation$RandomSeekScanTest 259.12 257.10 99.22%
          org.apache.hadoop.hbase.PerformanceEvaluation$RandomScanWithRange10Test 27966.00 27720.00 99.12%
          org.apache.hadoop.hbase.PerformanceEvaluation$RandomScanWithRange1000Test 2384.00 2364.00 99.16%
          Show
          Jean-Marc Spaggiari added a comment - Values for gets are running and will be posted very shortly. Test 0.94.12 patched 0.94.12 Diff org.apache.hadoop.hbase.PerformanceEvaluation$FilteredScanTest 10.62 10.55 99.34% org.apache.hadoop.hbase.PerformanceEvaluation$RandomSeekScanTest 259.12 257.10 99.22% org.apache.hadoop.hbase.PerformanceEvaluation$RandomScanWithRange10Test 27966.00 27720.00 99.12% org.apache.hadoop.hbase.PerformanceEvaluation$RandomScanWithRange1000Test 2384.00 2364.00 99.16%
          Hide
          Jean-Marc Spaggiari added a comment -

          With reads now:

            0.94.12 patched 0.94.12 Diff
          org.apache.hadoop.hbase.PerformanceEvaluation$FilteredScanTest 0.62 10.55 99.34%
          org.apache.hadoop.hbase.PerformanceEvaluation$RandomReadTest 817.73 824.11 100.78%
          org.apache.hadoop.hbase.PerformanceEvaluation$RandomSeekScanTest 259.12 257.10 99.22%
          org.apache.hadoop.hbase.PerformanceEvaluation$RandomScanWithRange10Test 27966.00 27720.00 99.12%
          org.apache.hadoop.hbase.PerformanceEvaluation$RandomScanWithRange1000Test 2384.00 2364.00 99.16%

          Basically, almost no difference between the 2. I'm running in StandAlone on SSD. I can run in pseudo-dist with 3 disks if you want.

          JM

          Show
          Jean-Marc Spaggiari added a comment - With reads now:   0.94.12 patched 0.94.12 Diff org.apache.hadoop.hbase.PerformanceEvaluation$FilteredScanTest 0.62 10.55 99.34% org.apache.hadoop.hbase.PerformanceEvaluation$RandomReadTest 817.73 824.11 100.78% org.apache.hadoop.hbase.PerformanceEvaluation$RandomSeekScanTest 259.12 257.10 99.22% org.apache.hadoop.hbase.PerformanceEvaluation$RandomScanWithRange10Test 27966.00 27720.00 99.12% org.apache.hadoop.hbase.PerformanceEvaluation$RandomScanWithRange1000Test 2384.00 2364.00 99.16% Basically, almost no difference between the 2. I'm running in StandAlone on SSD. I can run in pseudo-dist with 3 disks if you want. JM
          Hide
          Lars Hofhansl added a comment -

          Thanks JM! Lemme do one more test on my work machine (many cores and lots of RAM), will report back tomorrow or the day after. Then we should just commit this sucker.

          Show
          Lars Hofhansl added a comment - Thanks JM! Lemme do one more test on my work machine (many cores and lots of RAM), will report back tomorrow or the day after. Then we should just commit this sucker.
          Hide
          Jean-Marc Spaggiari added a comment -

          You're welcome.

          I can run it again with 32GB, or with a 4 node cluster, or with an 8 cores+32GB server or... or ... or...

          The only limitation are the disks. I have only one SSD, and can not have more than 6 disks per servers.

          All my servers are not running anything now, so all the options are correct. Just ask and I will run

          Show
          Jean-Marc Spaggiari added a comment - You're welcome. I can run it again with 32GB, or with a 4 node cluster, or with an 8 cores+32GB server or... or ... or... The only limitation are the disks. I have only one SSD, and can not have more than 6 disks per servers. All my servers are not running anything now, so all the options are correct. Just ask and I will run
          Hide
          Lars Hofhansl added a comment -

          Moving to 0.94.14. Want to do a few more tests, didn't get to it.
          Then we need to discuss 0.96 as well.

          Show
          Lars Hofhansl added a comment - Moving to 0.94.14. Want to do a few more tests, didn't get to it. Then we need to discuss 0.96 as well.
          Hide
          Jean-Marc Spaggiari added a comment -

          Just ping me if you need me to run tests again. Just build a 32GB server + SDD + SATA for Nick's tests

          Show
          Jean-Marc Spaggiari added a comment - Just ping me if you need me to run tests again. Just build a 32GB server + SDD + SATA for Nick's tests
          Hide
          Lars Hofhansl added a comment -

          Rebases 0.94 patch.
          TestReversibleScanners fails currently:

          Failed tests:   testReversibleMemstoreScanner(org.apache.hadoop.hbase.regionserver.TestReversibleScanners): expected:<testRow0199/testCf:testQual0001/1384370719137/Put/vlen=13/ts=0> but was:<testRow0199/testCf:testQual0000/1384370719137/Put/vlen=13/ts=7>
          
          Show
          Lars Hofhansl added a comment - Rebases 0.94 patch. TestReversibleScanners fails currently: Failed tests: testReversibleMemstoreScanner(org.apache.hadoop.hbase.regionserver.TestReversibleScanners): expected:<testRow0199/testCf:testQual0001/1384370719137/Put/vlen=13/ts=0> but was:<testRow0199/testCf:testQual0000/1384370719137/Put/vlen=13/ts=7>
          Hide
          chunhui shen added a comment -

          Rebases the trunk and 0.94 patch.

          TestReversibleScanners is failed because we change the action of MultiVersionConsistencyControl#setThreadReadPoint

          In v24 patch, I modify this test case and now it passes

          Show
          chunhui shen added a comment - Rebases the trunk and 0.94 patch. TestReversibleScanners is failed because we change the action of MultiVersionConsistencyControl#setThreadReadPoint In v24 patch, I modify this test case and now it passes
          Hide
          Lars Hofhansl added a comment -

          Thanks Chunhui. I tested together with Phoenix (forward scanning) only, and performance does not seem to be impacted.

          Show
          Lars Hofhansl added a comment - Thanks Chunhui. I tested together with Phoenix (forward scanning) only, and performance does not seem to be impacted.
          Hide
          James Taylor added a comment -

          Once this makes it into the 0.94 branch, we can make the necessary changes to Phoenix to do a reverse scan when otherwise we'd need to do an ORDER BY and let you know the performance difference. Thanks!

          Show
          James Taylor added a comment - Once this makes it into the 0.94 branch, we can make the necessary changes to Phoenix to do a reverse scan when otherwise we'd need to do an ORDER BY and let you know the performance difference. Thanks!
          Hide
          chunhui shen added a comment -

          Repost the patch for hadoop QA

          Show
          chunhui shen added a comment - Repost the patch for hadoop QA
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12613995/hbase-4811-trunkv24.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 18 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/7880//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12613995/hbase-4811-trunkv24.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 18 new or modified tests. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/7880//console This message is automatically generated.
          Hide
          chunhui shen added a comment -

          Rebases the patch

          Show
          chunhui shen added a comment - Rebases the patch
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12614178/hbase-4811-trunkv25.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 18 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          -1 javadoc. The javadoc tool appears to have generated 9 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          -1 site. The patch appears to cause mvn site goal to fail.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.regionserver.TestSplitTransaction

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12614178/hbase-4811-trunkv25.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 18 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. -1 javadoc . The javadoc tool appears to have generated 9 warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 -1 site . The patch appears to cause mvn site goal to fail. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestSplitTransaction Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/7894//console This message is automatically generated.
          Hide
          chunhui shen added a comment -

          I have run the TestSplitTransaction several times on local, all passed.

          Lars Hofhansl Ted Yu
          Is there any other comments or considerations? Is it OK to check in?

          Show
          chunhui shen added a comment - I have run the TestSplitTransaction several times on local, all passed. Lars Hofhansl Ted Yu Is there any other comments or considerations? Is it OK to check in?
          Hide
          Lars Hofhansl added a comment -

          The Phoenix guys volunteered to give this a try, test it a bit, including performance. James Taylor.
          Let's get it in in 0.94.15. Thanks for the perseverance chunhui shen, I know this is/was a bit painful.

          Show
          Lars Hofhansl added a comment - The Phoenix guys volunteered to give this a try, test it a bit, including performance. James Taylor . Let's get it in in 0.94.15. Thanks for the perseverance chunhui shen , I know this is/was a bit painful.
          Hide
          chunhui shen added a comment -

          Rebases the trunk patch

          Show
          chunhui shen added a comment - Rebases the trunk patch
          Hide
          chunhui shen added a comment -

          Lars Hofhansl
          Ok, I will wait the test result before commit it to 0.94.

          Show
          chunhui shen added a comment - Lars Hofhansl Ok, I will wait the test result before commit it to 0.94.
          Hide
          chunhui shen added a comment -

          Fix the failed test

          Show
          chunhui shen added a comment - Fix the failed test
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12614326/hbase-4811-trunkv27.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 18 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          -1 javadoc. The javadoc tool appears to have generated 10 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          -1 site. The patch appears to cause mvn site goal to fail.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12614326/hbase-4811-trunkv27.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 18 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. -1 javadoc . The javadoc tool appears to have generated 10 warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 -1 site . The patch appears to cause mvn site goal to fail. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/7908//console This message is automatically generated.
          Hide
          chunhui shen added a comment -

          Rebases the trunk patch

          Show
          chunhui shen added a comment - Rebases the trunk patch
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12614770/hbase-4811-trunkv28.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 18 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          -1 site. The patch appears to cause mvn site goal to fail.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.master.TestRestartCluster

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12614770/hbase-4811-trunkv28.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 18 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. -1 javadoc . The javadoc tool appears to have generated 1 warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. -1 findbugs . The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 -1 site . The patch appears to cause mvn site goal to fail. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.master.TestRestartCluster Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/7940//console This message is automatically generated.
          Hide
          Ted Yu added a comment -

          TestRestartCluster fails occasionally on trunk.
          The test failure is not related to latest patch.

          Show
          Ted Yu added a comment - TestRestartCluster fails occasionally on trunk. The test failure is not related to latest patch.
          Hide
          Ted Yu added a comment -

          +1 from me

          Show
          Ted Yu added a comment - +1 from me
          Hide
          Ted Yu added a comment -

          Rebased patch for QA run.

          Show
          Ted Yu added a comment - Rebased patch for QA run.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12616305/4811-trunk-v29.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 18 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          -1 site. The patch appears to cause mvn site goal to fail.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12616305/4811-trunk-v29.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 18 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 -1 site . The patch appears to cause mvn site goal to fail. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8027//console This message is automatically generated.
          Hide
          chunhui shen added a comment -

          Will commit this to trunk if no objection

          Show
          chunhui shen added a comment - Will commit this to trunk if no objection
          Hide
          chunhui shen added a comment -

          Commited to trunk,
          Thanks very much for the review, Ted, Lars,Jean-Marc,Andrew and others

          Show
          chunhui shen added a comment - Commited to trunk, Thanks very much for the review, Ted, Lars,Jean-Marc,Andrew and others
          Hide
          chunhui shen added a comment -

          Moving out of 0.94.15, wait lars to confirm when backport to 0.94

          Show
          chunhui shen added a comment - Moving out of 0.94.15, wait lars to confirm when backport to 0.94
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #859 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/859/)
          HBASE-4811 Support reverse Scan (zjushch: rev 1546878)

          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionServerCallable.java
          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedClientScanner.java
          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedScannerCallable.java
          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Scan.java
          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java
          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/Filter.java
          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FilterList.java
          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/PrefixFilter.java
          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
          • /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
          • /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java
          • /hbase/trunk/hbase-protocol/src/main/protobuf/Client.proto
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueHeap.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueScanner.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/NonReversedNonLazyKeyValueScanner.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedKeyValueHeap.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedRegionScannerImpl.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedStoreScanner.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/CollectionBackedScanner.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFilter.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/NoOpScanPolicyObserver.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
          Show
          Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #859 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/859/ ) HBASE-4811 Support reverse Scan (zjushch: rev 1546878) /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionServerCallable.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedClientScanner.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedScannerCallable.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Scan.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/Filter.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FilterList.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/PrefixFilter.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java /hbase/trunk/hbase-protocol/src/main/protobuf/Client.proto /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueHeap.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueScanner.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/NonReversedNonLazyKeyValueScanner.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedKeyValueHeap.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedRegionScannerImpl.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedStoreScanner.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/CollectionBackedScanner.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFilter.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/NoOpScanPolicyObserver.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in HBase-TRUNK #4705 (See https://builds.apache.org/job/HBase-TRUNK/4705/)
          HBASE-4811 Support reverse Scan (zjushch: rev 1546878)

          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionServerCallable.java
          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedClientScanner.java
          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedScannerCallable.java
          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Scan.java
          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java
          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/Filter.java
          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FilterList.java
          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/PrefixFilter.java
          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
          • /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
          • /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java
          • /hbase/trunk/hbase-protocol/src/main/protobuf/Client.proto
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueHeap.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueScanner.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/NonReversedNonLazyKeyValueScanner.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedKeyValueHeap.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedRegionScannerImpl.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedStoreScanner.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/CollectionBackedScanner.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFilter.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/NoOpScanPolicyObserver.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
          Show
          Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK #4705 (See https://builds.apache.org/job/HBase-TRUNK/4705/ ) HBASE-4811 Support reverse Scan (zjushch: rev 1546878) /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionServerCallable.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedClientScanner.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ReversedScannerCallable.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Scan.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/Filter.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FilterList.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/PrefixFilter.java /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java /hbase/trunk/hbase-protocol/src/main/protobuf/Client.proto /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueHeap.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueScanner.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/NonReversedNonLazyKeyValueScanner.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedKeyValueHeap.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedRegionScannerImpl.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedStoreScanner.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/CollectionBackedScanner.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFilter.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/NoOpScanPolicyObserver.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
          Hide
          Liang Xie added a comment -

          Glad to see it was in trunk now ! will revist our internal reverse scan bug fixed list to verify weather existing in trunk version or not, possible need to port a bit.

          Show
          Liang Xie added a comment - Glad to see it was in trunk now ! will revist our internal reverse scan bug fixed list to verify weather existing in trunk version or not, possible need to port a bit.
          Hide
          Lars Hofhansl added a comment -

          Cool.

          Show
          Lars Hofhansl added a comment - Cool.
          Hide
          Lars Hofhansl added a comment -

          Rebased 0.94 patch.

          Show
          Lars Hofhansl added a comment - Rebased 0.94 patch.
          Hide
          Lars Hofhansl added a comment -

          Attached the wrong patch. Will try again tomorrow.

          Show
          Lars Hofhansl added a comment - Attached the wrong patch. Will try again tomorrow.
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in HBase-TRUNK #4711 (See https://builds.apache.org/job/HBase-TRUNK/4711/)
          HBASE-10072. Regenerate ClientProtos after HBASE-4811 (apurtell: rev 1547720)

          • /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java
          Show
          Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK #4711 (See https://builds.apache.org/job/HBase-TRUNK/4711/ ) HBASE-10072 . Regenerate ClientProtos after HBASE-4811 (apurtell: rev 1547720) /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #863 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/863/)
          HBASE-10072. Regenerate ClientProtos after HBASE-4811 (apurtell: rev 1547720)

          • /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java
          Show
          Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #863 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/863/ ) HBASE-10072 . Regenerate ClientProtos after HBASE-4811 (apurtell: rev 1547720) /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java
          Hide
          Lars Hofhansl added a comment -

          Here's the rebased 0.94 patch.

          Show
          Lars Hofhansl added a comment - Here's the rebased 0.94 patch.

            People

            • Assignee:
              chunhui shen
              Reporter:
              John Carrino
            • Votes:
              2 Vote for this issue
              Watchers:
              32 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development