[HBASE-12731] Heap occupancy based client pushback

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0, 0.98.10, 1.1.0
• Hadoop Flags: Reviewed

Release Note

This feature incorporates reported regionserver heap occupancy in the (optional) client pushback calculations. If client pushback is enabled, the exponential backoff policy will take heap occupancy into account, should it exceed "hbase.heap.occupancy.low_water_mark" percentage of the heap (default 0.95). Once above the low water mark, heap occupancy is an additional factor scaling from 0.1 up to 1.0 at "hbase.heap.occupancy.high_water_mark" (default 0.98). At or above the high water mark the client will use the maximum configured backoff.

• Assignee: Andrew Kyle Purtell
• Reporter: Andrew Kyle Purtell
• Created: 19/Dec/14 19:56
• Updated: 06/Apr/18 17:55
• Resolved: 23/Jan/15 01:58

Description

If the heap occupancy of a RegionServer is beyond a configurable high water mark (suggestions: 95%, 98%) then we should reject all user RPCs and only allow administrative RPCs until occupancy has dropped below a configurable low water mark (suggestions: 92%).

Implement building on the HBASE-5162 changes.

It might be expensive to check heap occupancy, in which case we can sample it periodically with a chore and use the last known value in pushback calculations.

Issue Links

• relates to: HBASE-5162 Basic client pushback mechanism
• relates to: HBASE-14756 Break out ClientBackoffPolicy factors into configurable and composable components
• relates to: HBASE-12702 Backport client pushback