Details
-
New Feature
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
-
Adds a RegionServer config hbase.quota.user.override.key which can be set to the name of a request attribute whose value should be used as the username when evaluating quotas.
Description
The below is the original idea that started this work, but not what we actually landed on. See the first comment from rmdmattingly and the release note for that.
Old description:
Currently we provide the ability to define quotas for namespaces, tables, or users. On multi-tenant clusters, users may be broken down into groups based on their use-case. For us this comes down to 2 main cases:
- Hadoop jobs – it would be good to be able to limit all hadoop jobs in aggregate
- Proxy APIs - this is common where upstream callers don't hit hbase directly, instead they go through one of many proxy api's. For us we have a custom auth plugin which sets the username to the upstream caller name. But it would still be useful to be able to limit all usage from some particular proxy API in aggregate.
I think this could build upon the idea for Connection attributes in
HBASE-27657. Basically when a Connection is established we can set an attribute (i.e. quotaGrouping=hadoop or quotaGrouping=MyProxyAPI). In QuotaCache, we can add a getQuotaGroupLimiter(String groupName) and also allow someone to define quotas using set_quota TYPE => THROTTLE, GROUP => 'hadoop', LIMIT => '100M/sec'I need to do more investigation into whether we'd want to return a simple group limiter (more similar to table/namespace handling) or treat it more like the USER limiters which returns a QuotaState (so you can limit by-group-by-table).
We need to consider how GROUP quotas interact with USER quotas. If a user has a quota defined, and that user is also part of a group with a quota defined, does the request need to honor both quotas? Maybe we provide a GROUP_BYPASS setting, similar to GLOBAL_BYPASS?
Attachments
Issue Links
- links to