Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-8691

High-Throughput Streaming Scan API

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.95.0
    • Fix Version/s: None
    • Component/s: Scanners
    • Labels:

      Description

      I've done some working testing various ways to refactor and optimize Scans in HBase, and have found that performance can be dramatically increased by the addition of a streaming scan API. The attached code constitutes a proof of concept that shows performance increases of almost 4x in some workloads.

      I'd appreciate testing, replication, and comments. If the approach seems viable, I think such an API should be built into some future version of HBase.

        Attachments

        1. HRegionServlet.java
          0.3 kB
          Sandy Pratt
        2. README.txt
          8 kB
          Sandy Pratt
        3. RecordReceiver.java
          0.2 kB
          Sandy Pratt
        4. StreamHRegionServer.java
          1 kB
          Sandy Pratt
        5. StreamReceiverDirect.java
          4 kB
          Sandy Pratt
        6. ScannerTest.java
          5 kB
          Sandy Pratt
        7. StreamServletDirect.java
          3 kB
          Sandy Pratt

          Issue Links

            Activity

              People

              • Assignee:
                prattrs Sandy Pratt
                Reporter:
                prattrs Sandy Pratt
              • Votes:
                0 Vote for this issue
                Watchers:
                29 Start watching this issue

                Dates

                • Created:
                  Updated: