Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-13291

Lift the scan ceiling

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.0.0
    • Fix Version/s: None
    • Component/s: Scanners
    • Labels:
      None

      Description

      Scanning medium sized rows with multiple concurrent scanners exhibits interesting 'ceiling' properties. A server runs at about 6.7k ops a second using 450% of possible 1600% of CPUs when 4 clients each with 10 threads doing scan 1000 rows. If I add '--filterAll' argument (do not return results), then we run at 1450% of possible 1600% possible but we do 8k ops a second.

      Let me attach flame graphs for two cases. Unfortunately, there is some frustrating dark art going on. Let me try figure it... Filing issue in meantime to keep score in.

        Attachments

        1. traces.smaller.svg
          397 kB
          Michael Stack
        2. traces.small2.svg
          367 kB
          Michael Stack
        3. traces.nofilter.svg
          563 kB
          Michael Stack
        4. traces.filterall.svg
          341 kB
          Michael Stack
        5. traces.7.svg
          338 kB
          Michael Stack
        6. TimeRange.patch
          0.6 kB
          Anoop Sam John
        7. Screen Shot 2015-03-26 at 3.39.33 PM.png
          573 kB
          Michael Stack
        8. Screen Shot 2015-03-26 at 12.12.13 PM.png
          100 kB
          Michael Stack
        9. scan_no_mvcc_optimized.svg
          354 kB
          Michael Stack
        10. q (1).png
          10 kB
          Michael Stack
        11. nonBBposAndInineMvccVint.txt
          4 kB
          Michael Stack
        12. hack_to_bypass_bb.txt
          4 kB
          Michael Stack
        13. 13291.inlining.txt
          7 kB
          Michael Stack
        14. 13291.hacks.txt
          25 kB
          Michael Stack

          Issue Links

          relates to

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-5979 Non-pread DFSInputStreams should be associated with scanners, not HFile.Readers

          • Critical - Crashes, loss of data, severe memory leak.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-9102 HFile block pre-loading for large sequential scan

          • Major - Major loss of function.
          • Open

            Activity

              People

              • Assignee:
                stack Michael Stack
                Reporter:
                stack Michael Stack
              • Votes:
                0 Vote for this issue
                Watchers:
                20 Start watching this issue

                Dates

                • Created:
                  Updated: