Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-17877

Improve HBase's byte[] comparator

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.4.0, 1.3.2, 2.0.0
    • Component/s: util
    • Labels:
      None
    • Release Note:
      Hide
      updated the lexicographic byte array comparator to use a slightly more optimized version similar to the one available in the guava library that compares only the first index where left[index] != right[index]. The comparator also returns the diff directly instead of mapping it to -1, 0, +1 range as was being done in the earlier version. We have seen significant performance gains, calculated in terms of throughput (ops/ms) with these changes ranging from approx 20% for smaller byte arrays upto 200 bytes and almost 100% for large byte array sizes that are in few KB's. We benchmarked with upto 16KB arrays and the general trend indicates that the performance improvement increases as the size of the byte array increases.
      Show
      updated the lexicographic byte array comparator to use a slightly more optimized version similar to the one available in the guava library that compares only the first index where left[index] != right[index]. The comparator also returns the diff directly instead of mapping it to -1, 0, +1 range as was being done in the earlier version. We have seen significant performance gains, calculated in terms of throughput (ops/ms) with these changes ranging from approx 20% for smaller byte arrays upto 200 bytes and almost 100% for large byte array sizes that are in few KB's. We benchmarked with upto 16KB arrays and the general trend indicates that the performance improvement increases as the size of the byte array increases.

      Description

      Vikas Vishwakarma did some extensive tests and found that Hadoop's version is faster - dramatically faster in some cases.

      Patch forthcoming.

        Attachments

        1. HBASE-17877.branch-1.3.004.patch
          11 kB
          Vikas Vishwakarma
        2. HBASE-17877.branch-1.3.003.patch
          5 kB
          Vikas Vishwakarma
        3. HBASE-17877.master.003.patch
          8 kB
          Vikas Vishwakarma
        4. HBASE-17877.branch-1.3.002.patch
          4 kB
          Vikas Vishwakarma
        5. HBASE-17877.master.002.patch
          8 kB
          Vikas Vishwakarma
        6. HBASE-17877.master.001.patch
          7 kB
          Vikas Vishwakarma
        7. HBASE-17877.branch-1.3.001.patch
          4 kB
          Vikas Vishwakarma
        8. 17877-v4-1.3.patch
          4 kB
          Vikas Vishwakarma
        9. 17877-v3-1.3.patch
          4 kB
          Vikas Vishwakarma
        10. 17877-v2-1.3.patch
          3 kB
          Lars Hofhansl
        11. ByteComparatorJiraHBASE-17877.pdf
          190 kB
          Vikas Vishwakarma
        12. 17877-1.2.patch
          3 kB
          Lars Hofhansl

          Activity

            People

            • Assignee:
              vik.karma Vikas Vishwakarma
              Reporter:
              larsh Lars Hofhansl
            • Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: