From 228f572d62f712925e5c9fe9c3389fbc8dbc8355 Mon Sep 17 00:00:00 2001 From: Josh Elser Date: Wed, 12 Jun 2019 11:00:07 -0400 Subject: [PATCH] HBASE-22566 Update the 2.x upgrade chapter to include default flush/compaction throughput limits --- src/main/asciidoc/_chapters/upgrading.adoc | 35 ++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/src/main/asciidoc/_chapters/upgrading.adoc b/src/main/asciidoc/_chapters/upgrading.adoc index b20a2b25b6..3a3e588284 100644 --- a/src/main/asciidoc/_chapters/upgrading.adoc +++ b/src/main/asciidoc/_chapters/upgrading.adoc @@ -641,6 +641,41 @@ login from the configured keytab file and automatically refresh the credentials in the background for the process lifetime (See link:https://issues.apache.org/jira/browse/HBASE-16231[HBASE-16231]). +[[upgrade2.0.throughput.limits]] +.Default Throughput Controllers +HBase 2.x comes with default limits to the speed at which compactions and (Memstore) +flushes can execute. These limits are defined per RegionServer. In previous versions +of HBase, there was no limit to the speed at which a compaction or flush would run +by default. These configurations should ensure more stable operations from RegionServers. + +Take care to notice that these limits are _per RegionServer_, not _per compaction_ or _per flush_. + +Both throughput limits are defined as a range of bytes written per second, and is +allowed to vary within the given lower and upper bound. RegionServers observe the +current throughput and apply a linear formula to adjust the allowed throughput, within +the lower and upper bound, with respect to external pressure. For compactions, external +pressure is defined as the number of store files with respect to the maximum number +of allowed store files. For flushes, external pressure is defined as the current +Memstore size with respect to the maximum size of the Memstore. + +*Compactions* +- Default to a lower bound of `10 MB/s` and an upper bound of `20 MB/s`. +- These limits are controlled by `hbase.hstore.compaction.throughput.lower.bound` and + `hbase.hstore.compaction.throughput.higher.bound`. + +*Flushes* +- Default to a lower bound of `100 MB/s` and an upper bound of `200 MB/s`. +- These limits are controlled by `hbase.hstore.flush.throughput.lower.bound` and + `hbase.hstore.flush.throughput.upper.bound`. + +To revert to the unlimited throughput of earlier versions of HBase, please set +the respective property to the old implementation that applied no limits. + +*Compactions* +`hbase.regionserver.throughput.controller=org.apache.hadoop.hbase.regionserver.throttle.NoLimitThroughputController` + +*Flushes* +`hbase.regionserver.flush.throughput.controller=org.apache.hadoop.hbase.regionserver.throttle.NoLimitThroughputController` //// This would be a good place to link to an appendix on migrating applications -- 2.18.0