[HBASE-24628] Region normalizer now respects a rate limit

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha-1, 2.4.0
• Fix Version/s: 3.0.0-alpha-1, 2.4.0
• Component/s: Normalizer
• Hadoop Flags: Reviewed

Release Note

<!-- markdown -->
Introduces a new configuration, `hbase.normalizer.throughput.max_bytes_per_sec`, for specifying a limit on the throughput of actions executed by the normalizer. Note that while this configuration value is in bytes, the minimum honored valued is `1,000,000`, or `1m`. Supports values configured using the human-readable suffixes honored by [`Configuration.getLongBytes`](https://hadoop.apache.org/docs/current/api/org/apache/hadoop/conf/Configuration.html#getLongBytes-java.lang.String-long-)

• Assignee: Nick Dimiduk
• Reporter: Nick Dimiduk
• Created: 24/Jun/20 18:53
• Updated: 19/Oct/20 10:17
• Resolved: 19/Oct/20 10:17

Description

There's no limits on the normalizer right now. It will iterate through the tables one at a time until it's touched all tables. For any table, it generates a complete plan and executes that plan in totality.

It would be nice to allow operators to configure some limits here. Off the top of my head, the two metrics that might be interesting are split|merge actions / hour or hfile mb volume / hour. Either way, we'd need to track a little more metadata.

Issue Links

• breaks: HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky
• relates to: HBASE-24636 Increase default Normalizer interval
• links to: GitHub Pull Request #2454
• links to: GitHub Pull Request #2520
• links to: GitHub Pull Request #2540