Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20188

[TESTING] Performance

    XMLWordPrintableJSON

    Details

    • Type: Umbrella
    • Status: Open
    • Priority: Blocker
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 3.0.0, 2.3.0
    • Component/s: Performance
    • Labels:
      None

      Description

      How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor that it is much slower, that the problem is the asyncwal writing. Does in-memory compaction slow us down or speed us up? What happens when you enable offheaping?

      Keep notes here in this umbrella issue. Need to be able to say something about perf when 2.0.0 ships.

        Attachments

        1. flamegraph-1072.2.svg
          974 kB
          Michael Stack
        2. flamegraph-1072.1.svg
          1020 kB
          Michael Stack
        3. tree.txt
          985 kB
          Michael Stack
        4. ITBLL2.5B_1.2.7vs2.0.0_memheap.png
          81 kB
          Michael Stack
        5. ITBLL2.5B_1.2.7vs2.0.0_iops.png
          40 kB
          Michael Stack
        6. ITBLL2.5B_1.2.7vs2.0.0_gctime.png
          77 kB
          Michael Stack
        7. ITBLL2.5B_1.2.7vs2.0.0_ops.png
          56 kB
          Michael Stack
        8. ITBLL2.5B_1.2.7vs2.0.0_cpu.png
          165 kB
          Michael Stack
        9. ITBLL2.5B_1.2.7vs2.0.0_memstore.png
          23 kB
          Michael Stack
        10. ITBLL2.5B_1.2.7vs2.0.0_load.png
          112 kB
          Michael Stack
        11. ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png
          91 kB
          Michael Stack
        12. YCSB_GC_TIME.png
          24 kB
          Michael Stack
        13. YCSB_MEMSTORE.png
          29 kB
          Michael Stack
        14. YCSB_OPs.png
          32 kB
          Michael Stack
        15. YCSB_CPU.png
          46 kB
          Michael Stack
        16. YCSB_load.png
          42 kB
          Michael Stack
        17. YCSB_in-memory-compaction=NONE.ops.png
          28 kB
          Michael Stack
        18. YCSB_IN_MEMORY_COMPACTION=NONE.ops.png
          31 kB
          Michael Stack
        19. CAM-CONFIG-V01.patch
          3 kB
          Anastasia Braginsky
        20. HBase 2.0 performance evaluation - Basic vs None_ system settings.pdf
          36 kB
          Eshcar Hillel
        21. HBASE-20188.sh
          2 kB
          Eshcar Hillel
        22. lock.2.memsize2.c.20180403T160257Z.svg
          99 kB
          Michael Stack
        23. lock.127.workloadc.20180402T200918Z.svg
          63 kB
          Michael Stack
        24. hbase-site.xml
          8 kB
          Michael Stack
        25. run_ycsb.sh
          7 kB
          Michael Stack
        26. hbase-env.sh
          8 kB
          Michael Stack
        27. HBase 2.0 performance evaluation - 8GB.pdf
          36 kB
          Eshcar Hillel
        28. HBASE-20188-xac.sh
          2 kB
          Eshcar Hillel
        29. workloadx
          3 kB
          Eshcar Hillel
        30. hbase-site.xml
          3 kB
          Eshcar Hillel
        31. HBase 2.0 performance evaluation - 8GB(1).pdf
          58 kB
          Eshcar Hillel
        32. workloadx
          3 kB
          Eshcar Hillel
        33. total.png
          23 kB
          Michael Stack
        34. hits.png
          48 kB
          Michael Stack
        35. perregion.png
          48 kB
          Michael Stack
        36. hits_with_fp_scheduler.png
          33 kB
          Michael Stack
        37. HBase 2.0 performance evaluation - throughput SSD_HDD.pdf
          49 kB
          Eshcar Hillel

          Issue Links

          1.
          Expose in-memory compaction metrics Sub-task Open Anastasia Braginsky
          2.
          [locking] Write-time worst offenders Sub-task Patch Available Unassigned
          3.
          Update the doc on how to setup shortcircuit reads; its stale Sub-task Resolved Michael Stack
          4.
          IMC Default Parameters for 2.0.0 Sub-task Resolved Eshcar Hillel
          5.
          [DOC] Add note on perf to upgrade section Sub-task Resolved Michael Stack
          6.
          [IHC] Workloadx Sub-task Open Unassigned
          7.
          [PERFORMANCE] Flushing is 2x slower in hbase2. Sub-task Resolved Michael Stack
          8.
          Better heap utilization for IMC with MSLABs Sub-task Resolved Eshcar Hillel
          9.
          Tighter ByteBufferKeyValue Cell Comparator Sub-task Resolved Michael Stack
          10.
          Tighter ByteBufferKeyValue Cell Comparator; part 2 Sub-task Resolved Michael Stack
          11.
          SegmentScanner does over-comparing when one flushing Sub-task Resolved Michael Stack
          12.
          Reduce the number of Cell implementations; the profusion is distracting to users and JIT Sub-task In Progress Michael Stack
          13.
          extra cloneFamily() in Mutation.add(Cell) Sub-task Resolved Hua Xiang
          14.
          Save on a Cell iteration when writing Sub-task Patch Available Michael Stack
          15.
          ServerRpcConnection logging cleanup Sub-task Resolved Michael Stack
          16.
          Save recalculating families in a WALEdit batch of Cells Sub-task Resolved Michael Stack
          17.
          Table create with thousands of regions takes too long Sub-task Open Unassigned
          18.
          MemStoreLABImp::copyIntoCell uses 7% CPU when writing Sub-task Resolved Michael Stack
          19.
          Unsafe access cleanup Sub-task Resolved Sahil Aggarwal
          20.
          [amv2] large cluster startup is slow Sub-task Open Unassigned
          21.
          Remove all the CSLM#size operation in our memstore because it's an quite time consuming. Sub-task Resolved Zheng Hu
          22.
          Most of KeyValueUtil#length can be replaced by cell#getSerializedSize for better performance because the latter one has been optimized Sub-task Resolved Zheng Hu
          23.
          Size of in-memory compaction thread pool should be configurable Sub-task Resolved Zheng Hu

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                stack Michael Stack
              • Votes:
                0 Vote for this issue
                Watchers:
                32 Start watching this issue

                Dates

                • Created:
                  Updated: