[HBASE-2981] Improve consistency of performance during heavy load

• Type: Brainstorming
• Status: Closed
• Priority: Major
• Resolution: Incomplete
• Component/s: regionserver
• Assignee: Unassigned
• Reporter: Todd Lipcon
• Created: 10/Sep/10 17:26
• Updated: 12/Jun/22 00:27
• Resolved: 12/Jun/22 00:27

Description

Currently when running load tests like YCSB, we experience periods of complete cluster inactivity while all clients are blocked on a single region which is unavailable.

As discussed recently on the list, we should brainstorm some ideas for how to improve the situation: the goal here is to (a) minimize the amount of time when a region is inaccessible, and (b) minimize the impact that one inaccessible region has on other operations in the cluster.

Issue Links

• is related to: HBASE-6423 Writes should not block reads on blocking updates to memstores
• is related to: HBASE-5333 Introduce Memstore "backpressure" for writes
• relates to: HBASE-5162 Basic client pushback mechanism
• relates to: HBASE-2646 Compaction requests should be prioritized to prevent blocking
• relates to: HBASE-2158 Change how high/low global limit works; start taking on writes as soon as we dip below high limit rather than block until low limit as we currently do.
• relates to: HBASE-2782 QOS for META table access