[HBASE-13686] Fail to limit rate in RateLimiter

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.1.0, 2.0.0
• Fix Version/s: 1.2.0, 1.1.1, 2.0.0
• Component/s: Client
• Labels: multitenancy quota
• Hadoop Flags: Reviewed

Release Note

As per this jira contribution. We now support two kinds of RateLimiter.
1) org.apache.hadoop.hbase.quotas.AverageIntervalRateLimiter : This limiter will refill resources at every TimeUnit/resources interval.
Example: For a limiter configured with 10resources/second, then 1resource will be refilled after every 100ms.

2) org.apache.hadoop.hbase.quotas.FixedIntervalRateLimiter: This limiter will refill resources only after a given fixed interval of time.

Client can configure anyone of this rate limiter for the cluster by setting the value for the property "hbase.quota.rate.limiter" in the hbase-site.xml. org.apache.hadoop.hbase.quotas.AverageIntervalRateLimiter is the default value.
Note: Client needs to restart the cluster for the configuration to take into effect.

• Assignee: Ashish Singhi
• Reporter: Guanghao Zhang
• Due: 14/May/15
• Created: 14/May/15 03:53
• Updated: 24/Jun/22 18:55
• Resolved: 07/Jun/15 17:00

Description

While using the patch in HBASE-11598 , I found that RateLimiter can't to limit the rate right.

 
 /**
   * given the time interval, are there enough available resources to allow execution?
   * @param now the current timestamp
   * @param lastTs the timestamp of the last update
   * @param amount the number of required resources
   * @return true if there are enough available resources, otherwise false
   */
  public synchronized boolean canExecute(final long now, final long lastTs, final long amount) {
    return avail >= amount ? true : refill(now, lastTs) >= amount;
  }

When avail >= amount, avail can't be refill. But in the next time to call canExecute, lastTs maybe update. So avail will waste some time to refill. Even we use smaller rate than the limit, the canExecute will return false.

Issue Links

• is related to: HBASE-11598 Add simple rpc throttling