[HBASE-28012] Avoid CellUtil.cloneRow in BufferedEncodedSeeker

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.5.4
• Fix Version/s: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1
• Component/s: Offheaping, Performance
• Hadoop Flags: Reviewed
• Assignee: Becker Ewing
• Reporter: Becker Ewing
• Created: 09/Aug/23 02:06
• Updated: 20/Aug/23 06:53
• Resolved: 15/Aug/23 15:26

Description

A RegionServer running a workload that consisted of mostly reverse (and some forward) scans of thousands of tall (36 byte rows) under moderate read pressure (based on block bytes scanned) saw near 100% CPU usage. We noticed that the high CPU usage tended to correlate with more reverse scan quantity and the throughput of the server running more reverse scans was vastly lower than expectation. 

 

A flame graph of the Region Server at 100% CPU usage. Analysis of the flamegraph reveals that each seek for a reverse scan (which has more seeks than the forward seek path) has 2 CellUtil.cloneRow rowkey copies per seek. These CellUtil#cloneRow calls can be removed from this code path for off-heap cells in a similar fashion to HBASE-27146

Issue Links

• relates to: HBASE-28025 Enhance ByteBufferUtils.findCommonPrefix to compare 8 bytes each time
• links to: GitHub Pull Request #5347