[HBASE-21738] Remove all the CSLM#size operation in our memstore because it's an quite time consuming.

• Type: Sub-task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha-1, 2.2.0, 2.1.3, 2.0.5
• Component/s: Performance
• Hadoop Flags: Reviewed

Release Note

We found the memstore snapshotting would cost much time because of calling the time-consuming ConcurrentSkipListMap#Size, it would make the p999 latency spike happen. So in this issue, we remove all ConcurrentSkipListMap#size in memstore by counting the cellsCount in MemstoreSizeing. As the issue described, the p999 latency spike was mitigated.

• Assignee: Zheng Hu
• Reporter: Zheng Hu
• Created: 18/Jan/19 06:07
• Updated: 26/Jan/19 01:36
• Resolved: 21/Jan/19 10:11

Description

Made some performance test for 100% put case in branch-2 before.

We can see that there are many latency peak in p999 latency curve , and the peak time are almost the point time which our region is flushing.

See the hbase20-ssd-put-10000000000-rows-latencys-and-qps

And, I used the add-some-log.patch to log some time consuming when we grab the update.writeLock() to make a memstore snapshot. Tested again, I found those logs in log.txt.

Seems most of the time was consumed when taking memstore snapshot.. Let me dig into this.

Issue Links

• relates to: HBASE-21748 Port HBASE-21738 (Remove all the CLSM#size operation in our memstore because it's an quite time consuming.) to branch-1
• links to: Review board