[HBASE-20188] Evaluate and address performance delta between branch-1 and branch-2

• Type: Umbrella
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Component/s: Performance
• Assignee: Unassigned
• Reporter: Michael Stack
• Created: 13/Mar/18 19:11
• Updated: 07/Apr/20 23:30
• Resolved: 22/Jan/20 03:42

Description

How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor that it is much slower, that the problem is the asyncwal writing. Does in-memory compaction slow us down or speed us up? What happens when you enable offheaping?

Keep notes here in this umbrella issue. Need to be able to say something about perf when 2.0.0 ships.

Issue Links

• contains: HBASE-20710 extra cloneFamily() in Mutation.add(Cell)
• is related to: HBASE-21657 PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
• is related to: HBASE-21879 Read HFile's block to ByteBuffer directly instead of to byte for reducing young gc purpose
• relates to: HBASE-20337 Update the doc on how to setup shortcircuit reads; its stale
• relates to: HBASE-20411 Ameliorate MutableSegment synchronize
• relates to: HBASE-20457 Return immediately for a scan rpc call when we want to switch from pread to stream
• links to: 1.2.7 vs 2.0.0 Flush History during YCSB Load using Client-Side Batching
• links to: hbase1 to hbase2 compare with and without IN_MEMORY_COMPACTION and with and without MSLAB
• links to: Profling log/notes
• links to: Review Board (branch-2.0)

Sub-Tasks

• [locking] Write-time worst offenders: Resolved
• Update the doc on how to setup shortcircuit reads; its stale: Resolved
• IMC Default Parameters for 2.0.0: Resolved
• [DOC] Add note on perf to upgrade section: Resolved
• [IHC] Workloadx: Resolved
• [PERFORMANCE] Flushing is 2x slower in hbase2.: Resolved
• Better heap utilization for IMC with MSLABs: Resolved
• Tighter ByteBufferKeyValue Cell Comparator: Resolved
• Tighter ByteBufferKeyValue Cell Comparator; part 2: Resolved
• SegmentScanner does over-comparing when one flushing: Resolved
• extra cloneFamily() in Mutation.add(Cell): Resolved
• ServerRpcConnection logging cleanup: Resolved
• Save recalculating families in a WALEdit batch of Cells: Resolved
• MemStoreLABImp::copyIntoCell uses 7% CPU when writing: Resolved
• Unsafe access cleanup: Resolved
• Remove all the CSLM#size operation in our memstore because it's an quite time consuming.: Resolved
• Most of KeyValueUtil#length can be replaced by cell#getSerializedSize for better performance because the latter one has been optimized: Resolved
• Size of in-memory compaction thread pool should be configurable: Resolved