[HBASE-14969] Add throughput controller for flush

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0, 2.0.0
• Fix Version/s: 1.3.0, 2.0.0
• Component/s: regionserver
• Hadoop Flags: Reviewed

Release Note

Adds means of throttling flush throughput. By default there is no limit; we use NoLimitThroughputController. An alternative controller, PressureAwareFlushThroughputController, allows specifying throughput bounds. A new simple factor, flush pressure, influences throughput. See PressureAwareFlushThroughputController.java class for detail.

• Assignee: Yu Li
• Reporter: Yu Li
• Created: 13/Dec/15 06:49
• Updated: 28/Dec/17 17:54
• Resolved: 01/Feb/16 16:09

Description

In HBASE-8329 we added a throughput controller for compaction, to avoid spike caused by huge IO pressure like network/disk overflow. However, even with this control on, we are still observing disk utils near 100%, and by analysis we think this is caused by flush, especially when we increase the setting of hbase.hstore.flusher.count

In this JIRA, we propose to add throughput control feature for flush, as a supplement of HBASE-8329 to better control IO pressure.

Issue Links

• is related to: HBASE-15490 Remove duplicated CompactionThroughputControllerFactory in branch-1
• links to: Review board