Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-13291

Lift the scan ceiling

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.0.0
    • Fix Version/s: None
    • Component/s: Scanners
    • Labels:
      None

      Description

      Scanning medium sized rows with multiple concurrent scanners exhibits interesting 'ceiling' properties. A server runs at about 6.7k ops a second using 450% of possible 1600% of CPUs when 4 clients each with 10 threads doing scan 1000 rows. If I add '--filterAll' argument (do not return results), then we run at 1450% of possible 1600% possible but we do 8k ops a second.

      Let me attach flame graphs for two cases. Unfortunately, there is some frustrating dark art going on. Let me try figure it... Filing issue in meantime to keep score in.

        Attachments

        1. TimeRange.patch
          0.6 kB
          Anoop Sam John
        2. scan_no_mvcc_optimized.svg
          354 kB
          Michael Stack
        3. 13291.hacks.txt
          25 kB
          Michael Stack
        4. nonBBposAndInineMvccVint.txt
          4 kB
          Michael Stack
        5. Screen Shot 2015-03-26 at 3.39.33 PM.png
          573 kB
          Michael Stack
        6. hack_to_bypass_bb.txt
          4 kB
          Michael Stack
        7. traces.small2.svg
          367 kB
          Michael Stack
        8. Screen Shot 2015-03-26 at 12.12.13 PM.png
          100 kB
          Michael Stack
        9. q (1).png
          10 kB
          Michael Stack
        10. traces.7.svg
          338 kB
          Michael Stack
        11. 13291.inlining.txt
          7 kB
          Michael Stack
        12. traces.smaller.svg
          397 kB
          Michael Stack
        13. traces.filterall.svg
          341 kB
          Michael Stack
        14. traces.nofilter.svg
          563 kB
          Michael Stack

          Issue Links

            Activity

              People

              • Assignee:
                stack Michael Stack
                Reporter:
                stack Michael Stack
              • Votes:
                0 Vote for this issue
                Watchers:
                20 Start watching this issue

                Dates

                • Created:
                  Updated: