Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-7180

RegionScannerImpl.next() is inefficient.

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.94.4, 0.95.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We just came across a special scenario.

      For our Phoenix project (SQL runtime for HBase), we push a lot of work into HBase via coprocessors. One method is to wrap RegionScanner in coprocessor hooks and then do processing in the hook to avoid returning a lot of data to the client unnecessarily.

      In this specific case this is pretty bad. Since the wrapped RegionScanner's next() does not "know" that it is called this way is still does all of this on each invocation:

      1. Starts a RegionOperation
      2. Increments the request count
      3. set the current read point on a thread local (because generally each call could come from a different thread)
      4. Finally does the next on its StoreScanner(s)
      5. Ends the RegionOperation

      When this is done in a tight loop millions of times (as is the case for us) it starts to become significant.

      Not sure what to do about this, really. Opening this issue for discussion.

      One way is to extend the RegionScanner with an "internal" next() method of sorts, so that all this overhead can be avoided. The coprocessor could call the regular next() methods once and then just call the cheaper internal version.

      Are there better/cleaner ways?

        Attachments

        1. 7180-0.94-SKETCH.txt
          4 kB
          Lars Hofhansl
        2. 7180-0.94-v1.txt
          7 kB
          Lars Hofhansl
        3. 7180-0.94-v2.txt
          7 kB
          Lars Hofhansl
        4. 7180-0.94-v3.txt
          7 kB
          Lars Hofhansl
        5. 7180-0.96-v1.txt
          8 kB
          Lars Hofhansl
        6. 7180-0.94-v4.txt
          8 kB
          Lars Hofhansl
        7. 7180-0.96-v2.txt
          9 kB
          Lars Hofhansl
        8. 7180-0.96-v3.txt
          8 kB
          Lars Hofhansl
        9. 7180-0.94-v5.txt
          9 kB
          Lars Hofhansl
        10. 7180-0.96-v4.txt
          9 kB
          Lars Hofhansl

          Activity

            People

            • Assignee:
              lhofhansl Lars Hofhansl
              Reporter:
              lhofhansl Lars Hofhansl
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: