[HBASE-13291] Lift the scan ceiling

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Affects Version/s: 1.0.0
• Component/s: Scanners
• Assignee: Michael Stack
• Reporter: Michael Stack
• Created: 19/Mar/15 19:39
• Updated: 23/Jun/22 20:45
• Resolved: 11/Jun/22 18:23

Description

Scanning medium sized rows with multiple concurrent scanners exhibits interesting 'ceiling' properties. A server runs at about 6.7k ops a second using 450% of possible 1600% of CPUs when 4 clients each with 10 threads doing scan 1000 rows. If I add '--filterAll' argument (do not return results), then we run at 1450% of possible 1600% possible but we do 8k ops a second.

Let me attach flame graphs for two cases. Unfortunately, there is some frustrating dark art going on. Let me try figure it... Filing issue in meantime to keep score in.

Issue Links

• relates to: HBASE-5979 Non-pread DFSInputStreams should be associated with scanners, not HFile.Readers
• relates to: HBASE-9102 HFile block pre-loading for large sequential scan

Sub-Tasks

• Making methods under ScannerV2#next inlineable, faster: Closed
• [REGRESSION] HBASE-12600 undoes skip-mvcc parse optimizations: Closed
• Bypass logic in TimeRange.compare: Closed
• New Cell implementation with cached component offsets/lengths: Closed
• Make Bytes$LexicographicalComparerHolder$UnsafeComparer::compareTo inlineable: Closed
• Remove MVCC stamps from HFile when that is safe: Closed
• Avoid isCellTTLExpired() for NO-TAG cases: Closed
• Allow non KeyValue Cell types also to oswrite: Closed
• ByteBufferOuputStream - add writeInt support: Closed
• Make RPC layer CellBlock buffer a DirectByteBuffer: Closed
• Unsafe based optimized write in ByteBufferOutputStream: Closed
• Read mvcc vlong optimization: Closed
• Read path optimizations after HBASE-11425 profiling: Closed
• Reduce the number of time row comparison is done in a Scan: Closed