Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.98.0, 0.95.2
    • Component/s: Client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      OrderedBytes provides a data encoding format in which the resulting byte[] retains the same sort order as the natural types. Encoded formats can be inspected and decoded without forward knowledge of their content. Implementations are provided for integer and floating point numbers with 32- and 64-bits of precision, numeric values of arbitrary precision, Strings, and byte[]s. Utility methods for counting and skipping encoded entries are also provided.

      Most of the encoding formats are modeled after the SQLite4 key encoding format. Implementations of variable-length encodings are very similar. Fixed-width encodings are modeled after the fixed-width formats provided by the Orderly library. Javadocs on the OrderedBytes class describe the encoding format in detail. See http://sqlite.org/src4/doc/trunk/www/key_encoding.wiki for additional context.

      Notable deviation from the sqlite4 spec include:
       - Different header byte values. This is to give users more room to place their own, custom encoding extensions as they see fit.
       - BlobCopy is modified to include a termination byte of 0x00 in descending order. This is necessary in order to maintain reverse sort order of empty values. It is also renamed from "BlobLast".
       - Support for fixed-length integer and float encodings. Based on implementations from the Orderly library.
      Show
      OrderedBytes provides a data encoding format in which the resulting byte[] retains the same sort order as the natural types. Encoded formats can be inspected and decoded without forward knowledge of their content. Implementations are provided for integer and floating point numbers with 32- and 64-bits of precision, numeric values of arbitrary precision, Strings, and byte[]s. Utility methods for counting and skipping encoded entries are also provided. Most of the encoding formats are modeled after the SQLite4 key encoding format. Implementations of variable-length encodings are very similar. Fixed-width encodings are modeled after the fixed-width formats provided by the Orderly library. Javadocs on the OrderedBytes class describe the encoding format in detail. See http://sqlite.org/src4/doc/trunk/www/key_encoding.wiki for additional context. Notable deviation from the sqlite4 spec include:  - Different header byte values. This is to give users more room to place their own, custom encoding extensions as they see fit.  - BlobCopy is modified to include a termination byte of 0x00 in descending order. This is necessary in order to maintain reverse sort order of empty values. It is also renamed from "BlobLast".  - Support for fixed-length integer and float encodings. Based on implementations from the Orderly library.
    • Tags:
      0.96notable

      Description

      Once the spec is agreed upon, it must be implemented.

      1. 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch
        29 kB
        Nick Dimiduk
      2. 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch
        42 kB
        Nick Dimiduk
      3. 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch
        58 kB
        Nick Dimiduk
      4. 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch
        55 kB
        Nick Dimiduk
      5. 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch
        58 kB
        Nick Dimiduk
      6. 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch
        58 kB
        Nick Dimiduk
      7. 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch
        84 kB
        Nick Dimiduk
      8. 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch
        84 kB
        Nick Dimiduk
      9. 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch
        85 kB
        Nick Dimiduk
      10. 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch
        91 kB
        Nick Dimiduk
      11. 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch
        93 kB
        Nick Dimiduk
      12. 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch
        126 kB
        Nick Dimiduk
      13. 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch
        90 kB
        Nick Dimiduk
      14. 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch
        98 kB
        Nick Dimiduk
      15. 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch
        99 kB
        Nick Dimiduk
      16. 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch
        113 kB
        Nick Dimiduk
      17. 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch
        113 kB
        Nick Dimiduk
      18. 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch
        113 kB
        Nick Dimiduk

        Issue Links

          Activity

          Hide
          Nick Dimiduk added a comment -

          Hi Liangliang,

          We can. My preference from the beginning was to only provide the numeric encoding, because of the compatibility story. However, the fixed-length encodings seem popular. I created HBASE-9369 for this. I don't think it'll make 0.96.0RC0 though.

          Show
          Nick Dimiduk added a comment - Hi Liangliang, We can. My preference from the beginning was to only provide the numeric encoding, because of the compatibility story. However, the fixed-length encodings seem popular. I created HBASE-9369 for this. I don't think it'll make 0.96.0RC0 though.
          Hide
          He Liangliang added a comment -

          Hi Nick,
          Will fixed int8/byte and fixed int16/short be supported?

          Show
          He Liangliang added a comment - Hi Nick, Will fixed int8/byte and fixed int16/short be supported?
          Hide
          Hudson added a comment -

          FAILURE: Integrated in hbase-0.95-on-hadoop2 #235 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/235/)
          HBASE-8201 OrderedBytes: an ordered encoding strategy (Nick Dimiduck) (nkeywal: rev 1512924)

          • /hbase/branches/0.95/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Order.java
          • /hbase/branches/0.95/hbase-common/src/main/java/org/apache/hadoop/hbase/util/OrderedBytes.java
          • /hbase/branches/0.95/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestOrder.java
          • /hbase/branches/0.95/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestOrderedBytes.java
          Show
          Hudson added a comment - FAILURE: Integrated in hbase-0.95-on-hadoop2 #235 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/235/ ) HBASE-8201 OrderedBytes: an ordered encoding strategy (Nick Dimiduck) (nkeywal: rev 1512924) /hbase/branches/0.95/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Order.java /hbase/branches/0.95/hbase-common/src/main/java/org/apache/hadoop/hbase/util/OrderedBytes.java /hbase/branches/0.95/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestOrder.java /hbase/branches/0.95/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestOrderedBytes.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in hbase-0.95 #432 (See https://builds.apache.org/job/hbase-0.95/432/)
          HBASE-8201 OrderedBytes: an ordered encoding strategy (Nick Dimiduck) (nkeywal: rev 1512924)

          • /hbase/branches/0.95/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Order.java
          • /hbase/branches/0.95/hbase-common/src/main/java/org/apache/hadoop/hbase/util/OrderedBytes.java
          • /hbase/branches/0.95/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestOrder.java
          • /hbase/branches/0.95/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestOrderedBytes.java
          Show
          Hudson added a comment - SUCCESS: Integrated in hbase-0.95 #432 (See https://builds.apache.org/job/hbase-0.95/432/ ) HBASE-8201 OrderedBytes: an ordered encoding strategy (Nick Dimiduck) (nkeywal: rev 1512924) /hbase/branches/0.95/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Order.java /hbase/branches/0.95/hbase-common/src/main/java/org/apache/hadoop/hbase/util/OrderedBytes.java /hbase/branches/0.95/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestOrder.java /hbase/branches/0.95/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestOrderedBytes.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #668 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/668/)
          HBASE-8201 OrderedBytes: an ordered encoding strategy (Nick Dimiduck) (nkeywal: rev 1512923)

          • /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Order.java
          • /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/OrderedBytes.java
          • /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestOrder.java
          • /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestOrderedBytes.java
          Show
          Hudson added a comment - FAILURE: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #668 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/668/ ) HBASE-8201 OrderedBytes: an ordered encoding strategy (Nick Dimiduck) (nkeywal: rev 1512923) /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Order.java /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/OrderedBytes.java /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestOrder.java /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestOrderedBytes.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in HBase-TRUNK #4372 (See https://builds.apache.org/job/HBase-TRUNK/4372/)
          HBASE-8201 OrderedBytes: an ordered encoding strategy (Nick Dimiduck) (nkeywal: rev 1512923)

          • /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Order.java
          • /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/OrderedBytes.java
          • /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestOrder.java
          • /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestOrderedBytes.java
          Show
          Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #4372 (See https://builds.apache.org/job/HBase-TRUNK/4372/ ) HBASE-8201 OrderedBytes: an ordered encoding strategy (Nick Dimiduck) (nkeywal: rev 1512923) /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Order.java /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/OrderedBytes.java /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestOrder.java /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestOrderedBytes.java
          Hide
          stack added a comment -

          +1

          Show
          stack added a comment - +1
          Hide
          Nicolas Liochon added a comment -

          It does not compile because it now depends on HBASE-9091. The previous versions were +1ed. Any issue with this version? If not, and if the other jiras on types (HBASE-9091 and HBASE-8693) make it I will commit this week-end.

          Show
          Nicolas Liochon added a comment - It does not compile because it now depends on HBASE-9091 . The previous versions were +1ed. Any issue with this version? If not, and if the other jiras on types ( HBASE-9091 and HBASE-8693 ) make it I will commit this week-end.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12596749/0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 8 new or modified tests.

          -1 hadoop1.0. The patch failed to compile against the hadoop 1.0 profile.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6647//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12596749/0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 8 new or modified tests. -1 hadoop1.0 . The patch failed to compile against the hadoop 1.0 profile. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6647//console This message is automatically generated.
          Hide
          Nick Dimiduk added a comment -

          Addressing Ted's other comments I missed the first time.

          Show
          Nick Dimiduk added a comment - Addressing Ted's other comments I missed the first time.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12596739/0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 8 new or modified tests.

          -1 hadoop1.0. The patch failed to compile against the hadoop 1.0 profile.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6644//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12596739/0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 8 new or modified tests. -1 hadoop1.0 . The patch failed to compile against the hadoop 1.0 profile. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6644//console This message is automatically generated.
          Hide
          Nick Dimiduk added a comment -

          Addressing Ted's comments from RB.

          Show
          Nick Dimiduk added a comment - Addressing Ted's comments from RB.
          Hide
          Nick Dimiduk added a comment -

          Updated patch to use ByteRange.

          Show
          Nick Dimiduk added a comment - Updated patch to use ByteRange.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12595842/0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 8 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6602//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6602//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6602//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6602//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6602//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6602//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6602//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6602//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6602//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6602//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12595842/0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 8 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6602//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6602//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6602//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6602//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6602//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6602//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6602//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6602//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6602//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6602//console This message is automatically generated.
          Hide
          Nick Dimiduk added a comment -

          Fix javadoc warnings.

          Show
          Nick Dimiduk added a comment - Fix javadoc warnings.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12595833/0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 8 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          -1 javadoc. The javadoc tool appears to have generated 2 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6599//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6599//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6599//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6599//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6599//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6599//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6599//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6599//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6599//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6599//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12595833/0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 8 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. -1 javadoc . The javadoc tool appears to have generated 2 warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6599//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6599//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6599//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6599//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6599//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6599//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6599//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6599//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6599//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6599//console This message is automatically generated.
          Hide
          Nick Dimiduk added a comment -

          Addressing more reviewer comments.

          • fixes the long -> int cast
          • removes Numeric helper class, leaving decoded object instantiation up to clients
          • removes conditional logic from Order, adds tests
          • beefs up documentation on OrderedBytes, and corrects handling of 0x00 byte in BlobCopy ASCENDING
          • adds helper functions for inspecting the nature of encoded values
          Show
          Nick Dimiduk added a comment - Addressing more reviewer comments. fixes the long -> int cast removes Numeric helper class, leaving decoded object instantiation up to clients removes conditional logic from Order, adds tests beefs up documentation on OrderedBytes, and corrects handling of 0x00 byte in BlobCopy ASCENDING adds helper functions for inspecting the nature of encoded values
          Hide
          Nick Dimiduk added a comment -

          => IntelliJ says it does nothing and I think is right?

          Eclipse doesn't warn me about such constructs

          I think you are correct. This is an explicit truncating cast from long to int, and should be the same as

          y = (int) val;
          

          I ran the tests before and after, green all the way. Feel free to make this change on commit

          Show
          Nick Dimiduk added a comment - => IntelliJ says it does nothing and I think is right? Eclipse doesn't warn me about such constructs I think you are correct. This is an explicit truncating cast from long to int , and should be the same as y = (int) val; I ran the tests before and after, green all the way. Feel free to make this change on commit
          Hide
          Nicolas Liochon added a comment -

          I read the code, it seems great. I would have personally like the integration with byteRange but it can be done later.
          My intelliJ complains about something that can be fixed on commit:

          y = (int) (val & 0xffffffff);

          => IntelliJ says it does nothing and I think is right?

          +1 anyway. It seems there is a +1 from Sergey Shelukhin and Ted Yu was ok, so I believe I can commit. Will do that in a few hours if nobody objects.

          Show
          Nicolas Liochon added a comment - I read the code, it seems great. I would have personally like the integration with byteRange but it can be done later. My intelliJ complains about something that can be fixed on commit: y = (int) (val & 0xffffffff); => IntelliJ says it does nothing and I think is right? +1 anyway. It seems there is a +1 from Sergey Shelukhin and Ted Yu was ok, so I believe I can commit. Will do that in a few hours if nobody objects.
          Hide
          Matt Corgan added a comment -

          unofficial +1 from me

          Show
          Matt Corgan added a comment - unofficial +1 from me
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12595533/0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6565//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6565//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12595533/0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6565//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6565//console This message is automatically generated.
          Hide
          Nick Dimiduk added a comment -

          I should add, this version provides Order and OrderedBytes, no ordered data types. Now on RB.

          Show
          Nick Dimiduk added a comment - I should add, this version provides Order and OrderedBytes, no ordered data types. Now on RB.
          Hide
          Nick Dimiduk added a comment -

          One more time. The API in this version has no dependencies – all methods take a byte[] buff and int offset as first two arguments. Per my previous comment, that means the client must pass over an encoded value twice: once to get the encoded value, and again to advance the offset.

          Show
          Nick Dimiduk added a comment - One more time. The API in this version has no dependencies – all methods take a byte[] buff and int offset as first two arguments. Per my previous comment, that means the client must pass over an encoded value twice: once to get the encoded value, and again to advance the offset .
          Hide
          Nick Dimiduk added a comment -

          I'm looking for a way to remove dependencies from this patch. Please see this comment comment, it should have been on this ticket rather than the other one.

          Show
          Nick Dimiduk added a comment - I'm looking for a way to remove dependencies from this patch. Please see this comment comment , it should have been on this ticket rather than the other one.
          Hide
          Nick Dimiduk added a comment -

          Restructured to provide OrderedBytes encoding + DataType implementations. Now depends on HBASE-9091 + HBASE-8693.

          Show
          Nick Dimiduk added a comment - Restructured to provide OrderedBytes encoding + DataType implementations. Now depends on HBASE-9091 + HBASE-8693 .
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12593611/0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6430//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6430//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6430//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6430//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6430//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6430//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6430//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6430//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6430//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6430//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12593611/0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6430//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6430//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6430//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6430//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6430//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6430//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6430//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6430//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6430//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6430//console This message is automatically generated.
          Hide
          Nick Dimiduk added a comment -

          This patch addresses comments from JIRA and RB, including

          • rename blobmid, bloblast to blobvar, blobcopy
          • refactor blobvar to not require a term byte in most cases
          • extend documentation around blobvar encoding
          • rename HNumeric -> Numeric
          Show
          Nick Dimiduk added a comment - This patch addresses comments from JIRA and RB, including rename blobmid, bloblast to blobvar, blobcopy refactor blobvar to not require a term byte in most cases extend documentation around blobvar encoding rename HNumeric -> Numeric
          Hide
          Ted Yu added a comment -

          Blob-mid encoding is referenced by HBASE-8693

          Please add javadoc in the patch to explain how it works.

          Show
          Ted Yu added a comment - Blob-mid encoding is referenced by HBASE-8693 Please add javadoc in the patch to explain how it works.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12591885/0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6310//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6310//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6310//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6310//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6310//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6310//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6310//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6310//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6310//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6310//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12591885/0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6310//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6310//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6310//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6310//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6310//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6310//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6310//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6310//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6310//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6310//console This message is automatically generated.
          Hide
          Nick Dimiduk added a comment -

          This patch addresses the release audit warning, adds test coverage for OrderedBytes#skip, and make a couple tweaks informed by further work on HBASE-8693.

          Show
          Nick Dimiduk added a comment - This patch addresses the release audit warning, adds test coverage for OrderedBytes#skip, and make a couple tweaks informed by further work on HBASE-8693 .
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12591742/0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          -1 release audit. The applied patch generated 1 release audit warnings (more than the trunk's current 0 warnings).

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.TestFullLogReconstruction

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12591742/0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. -1 release audit . The applied patch generated 1 release audit warnings (more than the trunk's current 0 warnings). +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.TestFullLogReconstruction Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6303//console This message is automatically generated.
          Hide
          Nick Dimiduk added a comment -

          This patch addresses Sergey Shelukhin's comments about isolation of complement logic. It makes use of Order#apply, rather than allowing mask implementation details to permeate. It also replaces header byte literals with named constants. Hopefully these changes make the code easier to read and the localized logic simpler to follow.

          Show
          Nick Dimiduk added a comment - This patch addresses Sergey Shelukhin 's comments about isolation of complement logic. It makes use of Order#apply , rather than allowing mask implementation details to permeate. It also replaces header byte literals with named constants. Hopefully these changes make the code easier to read and the localized logic simpler to follow.
          Hide
          Nick Dimiduk added a comment -

          Yeah. For unsigned shift right in Java, you want >>>.

          Show
          Nick Dimiduk added a comment - Yeah. For unsigned shift right in Java, you want >>> .
          Hide
          Sergey Shelukhin added a comment -

          The fn call looks better, I was thinking more along the lines of calling something on Order that would do both assert and return the correct order, not sure about performance implications of that...

          As for bits, nevermind, I am not used to bit operations on signed types, didn't realize >> won't fill with zeros but rather with a sign bit

          Show
          Sergey Shelukhin added a comment - The fn call looks better, I was thinking more along the lines of calling something on Order that would do both assert and return the correct order, not sure about performance implications of that... As for bits, nevermind, I am not used to bit operations on signed types, didn't realize >> won't fill with zeros but rather with a sign bit
          Hide
          Nick Dimiduk added a comment -

          This kind of code makes order encapsulation leak.

          Agreed. How about something like the following?

          assert header == 0x30 || header == Order.DESCENDING.apply((byte) 0x30);
          

          I am not getting something here...

          The idea is to create a mask that supports sort order preservation for negative numbers while also accounting for the range of negative values being one larger than the range of positive values. From the spec,

          int: The int data type is a 32-bit signed two's complement integer. It has a minimum value of -2,147,483,648 and a maximum value of 2,147,483,647 (inclusive)

          So this logic creates a mask that will invert the sign bit and maintain the range.

              for (int i : new int[] { 5, -5, Integer.MAX_VALUE, Integer.MIN_VALUE }) {
                int j = (i >> Integer.SIZE - 1) | Integer.MIN_VALUE;
                int k = i ^ j;
                System.out.println("Input:   0x" + Integer.toHexString(i));
                System.out.println("Shifted: 0x" + Integer.toHexString((i >> Integer.SIZE - 1)));
                System.out.println("MIN_VAL: 0x" + Integer.toHexString(Integer.MIN_VALUE));
                System.out.println("Mask:    0x" + Integer.toHexString(j));
                System.out.println("Result:  0x" + Integer.toHexString(k));
                System.out.println();
              }
          
          Input:   0x5
          Shifted: 0x0
          MIN_VAL: 0x80000000
          Mask:    0x80000000
          Result:  0x80000005
          
          Input:   0xfffffffb
          Shifted: 0xffffffff
          MIN_VAL: 0x80000000
          Mask:    0xffffffff
          Result:  0x4
          
          Input:   0x7fffffff
          Shifted: 0x0
          MIN_VAL: 0x80000000
          Mask:    0x80000000
          Result:  0xffffffff
          
          Input:   0x80000000
          Shifted: 0xffffffff
          MIN_VAL: 0x80000000
          Mask:    0xffffffff
          Result:  0x7fffffff
          
          Show
          Nick Dimiduk added a comment - This kind of code makes order encapsulation leak. Agreed. How about something like the following? assert header == 0x30 || header == Order.DESCENDING.apply((byte) 0x30); I am not getting something here... The idea is to create a mask that supports sort order preservation for negative numbers while also accounting for the range of negative values being one larger than the range of positive values. From the spec , int : The int data type is a 32-bit signed two's complement integer. It has a minimum value of -2,147,483,648 and a maximum value of 2,147,483,647 (inclusive) So this logic creates a mask that will invert the sign bit and maintain the range. for ( int i : new int [] { 5, -5, Integer .MAX_VALUE, Integer .MIN_VALUE }) { int j = (i >> Integer .SIZE - 1) | Integer .MIN_VALUE; int k = i ^ j; System .out.println( "Input: 0x" + Integer .toHexString(i)); System .out.println( "Shifted: 0x" + Integer .toHexString((i >> Integer .SIZE - 1))); System .out.println( "MIN_VAL: 0x" + Integer .toHexString( Integer .MIN_VALUE)); System .out.println( "Mask: 0x" + Integer .toHexString(j)); System .out.println( "Result: 0x" + Integer .toHexString(k)); System .out.println(); } Input: 0x5 Shifted: 0x0 MIN_VAL: 0x80000000 Mask: 0x80000000 Result: 0x80000005 Input: 0xfffffffb Shifted: 0xffffffff MIN_VAL: 0x80000000 Mask: 0xffffffff Result: 0x4 Input: 0x7fffffff Shifted: 0x0 MIN_VAL: 0x80000000 Mask: 0x80000000 Result: 0xffffffff Input: 0x80000000 Shifted: 0xffffffff MIN_VAL: 0x80000000 Mask: 0xffffffff Result: 0x7fffffff
          Hide
          Sergey Shelukhin added a comment -
              assert header == 0x30 || header == (byte) 0xCF;
              Order ord = header == 0x30 ? Order.ASCENDING : Order.DESCENDING;
          

          This kind of code makes order encapsulation leak. But I guess it's better than a method call to order method with base value.

          i ^= ((i >> Integer.SIZE - 1) | Integer.MIN_VALUE);
          

          I am not getting something here... The sign bit seems to be shifted into the rightmost bit, and then or-ed with the mask with leftmost bit set? So, for negative number, the mask to xor with will look like 1000...0001? Am I missing something?

          Show
          Sergey Shelukhin added a comment - assert header == 0x30 || header == ( byte ) 0xCF; Order ord = header == 0x30 ? Order.ASCENDING : Order.DESCENDING; This kind of code makes order encapsulation leak. But I guess it's better than a method call to order method with base value. i ^= ((i >> Integer .SIZE - 1) | Integer .MIN_VALUE); I am not getting something here... The sign bit seems to be shifted into the rightmost bit, and then or-ed with the mask with leftmost bit set? So, for negative number, the mask to xor with will look like 1000...0001? Am I missing something?
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12591455/0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          -1 release audit. The applied patch generated 1 release audit warnings (more than the trunk's current 0 warnings).

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12591455/0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. -1 release audit . The applied patch generated 1 release audit warnings (more than the trunk's current 0 warnings). +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/6276//console This message is automatically generated.
          Hide
          Nick Dimiduk added a comment -

          Improvements based on experience implementing a couple data types over on HBASE-8693. Also added fixed-width int and float encodings after conversation with Enis Soztutar. Summary of changes includes:

          • consolidate on Numeric and BigDecimal for lossless en/decoding
          • refactor Order, HNumeric into separate classes
          • extend Numerics tests
          • add fixed-length encodings for int and float
          • better docstrings
          • save a byte on ASCENDING BlobLast encoding
          • improve test coverage around blobMid encoding
          • move to hbase-common module
          Show
          Nick Dimiduk added a comment - Improvements based on experience implementing a couple data types over on HBASE-8693 . Also added fixed-width int and float encodings after conversation with Enis Soztutar . Summary of changes includes: consolidate on Numeric and BigDecimal for lossless en/decoding refactor Order, HNumeric into separate classes extend Numerics tests add fixed-length encodings for int and float better docstrings save a byte on ASCENDING BlobLast encoding improve test coverage around blobMid encoding move to hbase-common module
          Hide
          Sergey Shelukhin added a comment -

          +1

          Patch is fairly involved, not sure if more reviews are necessary.
          Moving to common makes sense.
          Does it make sense to compute the epsilon?

          Show
          Sergey Shelukhin added a comment - +1 Patch is fairly involved, not sure if more reviews are necessary. Moving to common makes sense. Does it make sense to compute the epsilon?
          Hide
          Nick Dimiduk added a comment -

          Fix bug that was missed by build env discrepancy.

          Show
          Nick Dimiduk added a comment - Fix bug that was missed by build env discrepancy.
          Hide
          Nick Dimiduk added a comment -

          I'd also like to propose moving OrderedBytes into hbase-common module. I think this functionality will be necessary for implementing Coprocessors, Filters and atomic server-side data manipulations to take advantage of the encoding scheme. My proposal maintins the Data Type definitions remain in hbase-client.

          Show
          Nick Dimiduk added a comment - I'd also like to propose moving OrderedBytes into hbase-common module. I think this functionality will be necessary for implementing Coprocessors, Filters and atomic server-side data manipulations to take advantage of the encoding scheme. My proposal maintins the Data Type definitions remain in hbase-client.
          Hide
          Nick Dimiduk added a comment -

          Fixes floating-point error through use of BigDecimal for value encoding. Consolidates code-paths. Extends test coverage. Addresses Sergey Shelukhin's latest comments. On rb.

          Show
          Nick Dimiduk added a comment - Fixes floating-point error through use of BigDecimal for value encoding. Consolidates code-paths. Extends test coverage. Addresses Sergey Shelukhin 's latest comments. On rb.
          Hide
          Nick Dimiduk added a comment -

          New patch cleaned up for reviewbot and addresses a couple of Sergey Shelukhin's comments.

          Show
          Nick Dimiduk added a comment - New patch cleaned up for reviewbot and addresses a couple of Sergey Shelukhin 's comments.
          Hide
          Sergey Shelukhin added a comment -

          left some feedback on rb

          Show
          Sergey Shelukhin added a comment - left some feedback on rb
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12586590/0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified tests.

          +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          -1 javadoc. The javadoc tool appears to have generated 161 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.TestIOFencing

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5964//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5964//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5964//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5964//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5964//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5964//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5964//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5964//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5964//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5964//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12586590/0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified tests. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. -1 javadoc . The javadoc tool appears to have generated 161 warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.TestIOFencing Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5964//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5964//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5964//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5964//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5964//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5964//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5964//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5964//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5964//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5964//console This message is automatically generated.
          Hide
          Nick Dimiduk added a comment -

          Patch 3 attached and on RB.

          This version tightens up the tests to be more careful about buffer over/under-run and also ensure reading/decoding an encoded buffer does not modify underlying byte[]. It also implements a couple utility methods for working with encoded values. The whole thing is up for a proper performance evaluation, for which I opened HBASE-8694.

          A notable deviation from the sqlite spec is in the blob-last format. Blob-last is modified to include a termination byte of 0x00. This is necessary in order to maintain reverse sort order of empty values.

          A notable difference between this specification and Java is the sort order of NaN. This spec treats NaN as less than all values, including null. Java sorts NaN as greater than any floating point value.
          See http://docs.oracle.com/javase/1.5.0/docs/api/java/util/Arrays.html#sort(double[])

          Major TODO: The numeric encoding and decoding implementation suffers from floating point rounding errors. I'm going to read a Goldberg paper and hope for a way around this, so in the mean time some of the test scenarios are commented out.

          Show
          Nick Dimiduk added a comment - Patch 3 attached and on RB. This version tightens up the tests to be more careful about buffer over/under-run and also ensure reading/decoding an encoded buffer does not modify underlying byte[]. It also implements a couple utility methods for working with encoded values. The whole thing is up for a proper performance evaluation, for which I opened HBASE-8694 . A notable deviation from the sqlite spec is in the blob-last format. Blob-last is modified to include a termination byte of 0x00. This is necessary in order to maintain reverse sort order of empty values. A notable difference between this specification and Java is the sort order of NaN. This spec treats NaN as less than all values, including null. Java sorts NaN as greater than any floating point value. See http://docs.oracle.com/javase/1.5.0/docs/api/java/util/Arrays.html#sort(double[ ]) Major TODO: The numeric encoding and decoding implementation suffers from floating point rounding errors. I'm going to read a Goldberg paper and hope for a way around this, so in the mean time some of the test scenarios are commented out.
          Hide
          Ted Yu added a comment -

          More interesting would be comparing this to Orderly or Phoenix type implementation

          Agreed. Comparison with Phoenix type implementation can be done first since Phoenix is actively maintained.

          Show
          Ted Yu added a comment - More interesting would be comparing this to Orderly or Phoenix type implementation Agreed. Comparison with Phoenix type implementation can be done first since Phoenix is actively maintained.
          Hide
          Nick Dimiduk added a comment -

          I've not done any performance benchmark yet. Besides, comparing this to Bytes is not interesting because they provide different functionality. More interesting would be comparing this to Orderly or Phoenix type implementation. I'll work something out after the next patch.

          Show
          Nick Dimiduk added a comment - I've not done any performance benchmark yet. Besides, comparing this to Bytes is not interesting because they provide different functionality. More interesting would be comparing this to Orderly or Phoenix type implementation. I'll work something out after the next patch.
          Hide
          Matteo Bertozzi added a comment -

          Do you have a simple benchmark to see how much overhead the OrderedBytes methods (put, get, compare) have compared to the Bytes.*?

          Show
          Matteo Bertozzi added a comment - Do you have a simple benchmark to see how much overhead the OrderedBytes methods (put, get, compare) have compared to the Bytes.*?
          Show
          Nick Dimiduk added a comment - https://reviews.apache.org/r/11633/
          Hide
          Ted Yu added a comment -

          The patch is not small.

          Mind putting it on review board ?

          Show
          Ted Yu added a comment - The patch is not small. Mind putting it on review board ?
          Hide
          Nick Dimiduk added a comment -

          This patch addresses:

          • encoding and decoding sequences of values ("compound rowkey")
          • bounds/error checking with String encoding
          • support for "mid" blob encoding and decoding, allowing byte[] to be used in the middle of compound keys.

          Remaining TODOs include:

          • descending sort order
          • utilities for counting, skipping encoded values on a ByteBuffer
          • consider using BigInteger/BigDecimal to allow for arbitrary size and precision numerics.

          (CC Matt Corgan, James Taylor)

          Show
          Nick Dimiduk added a comment - This patch addresses: encoding and decoding sequences of values ("compound rowkey") bounds/error checking with String encoding support for "mid" blob encoding and decoding, allowing byte[] to be used in the middle of compound keys. Remaining TODOs include: descending sort order utilities for counting, skipping encoded values on a ByteBuffer consider using BigInteger/BigDecimal to allow for arbitrary size and precision numerics. (CC Matt Corgan , James Taylor )
          Hide
          Nick Dimiduk added a comment -

          WIP Commit. OrderedBytes provides a serialization format in which the resulting byte[] retains the same sort order as the natural types. Serialized formats can be inspected and decoded without forward knowledge of their content. Implementations are provided for Numeric values with up to 64-bits of precision, Strings, and byte[].

          The encoding format is modeled after the SQLite4 key encoding format. See http://sqlite.org/src4/doc/trunk/www/key_encoding.wiki for details.

          TODOs include:

          • descending sort order
          • encoding and decoding sequences of values ("compound rowkey")
          • bounds/error checking with String encoding
          • support for "mid" blob encoding and decoding, allowing byte[] to be used in the middle of compound keys.
          • utilities for counting, skipping encoded values on a ByteBuffer
          • consider using BigInteger/BigDecimal to allow for arbitrary size and precision numerics.
          Show
          Nick Dimiduk added a comment - WIP Commit. OrderedBytes provides a serialization format in which the resulting byte[] retains the same sort order as the natural types. Serialized formats can be inspected and decoded without forward knowledge of their content. Implementations are provided for Numeric values with up to 64-bits of precision, Strings, and byte[]. The encoding format is modeled after the SQLite4 key encoding format. See http://sqlite.org/src4/doc/trunk/www/key_encoding.wiki for details. TODOs include: descending sort order encoding and decoding sequences of values ("compound rowkey") bounds/error checking with String encoding support for "mid" blob encoding and decoding, allowing byte[] to be used in the middle of compound keys. utilities for counting, skipping encoded values on a ByteBuffer consider using BigInteger/BigDecimal to allow for arbitrary size and precision numerics.
          Hide
          Nick Dimiduk added a comment -

          My work on implementing strategies documented is available here: https://github.com/ndimiduk/serialization-play

          Show
          Nick Dimiduk added a comment - My work on implementing strategies documented is available here: https://github.com/ndimiduk/serialization-play

            People

            • Assignee:
              Nick Dimiduk
              Reporter:
              Nick Dimiduk
            • Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development