[HBASE-27048] Server side scanner time limit should account for time in queue

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.5.0, 3.0.0-alpha-4, 2.4.14
• Release Note: Server will now account for queue time when determining how long a scanner can run before heartbeat should be returned. This should help avoid timeouts when server is overloaded.
• Assignee: Bryan Beaudreault
• Reporter: Bryan Beaudreault
• Created: 17/May/22 16:16
• Updated: 16/Jul/22 18:42
• Resolved: 14/Jul/22 23:34

Description

When a scan request comes in with a timeout specified and heartbeats/partials allowed, we calculate a time limit for running the scan to be half of that timeout. The idea is to return before the timeout expires.

The calculation of that time limit is "now + timeout / 2", where now is the point at which the scan is starting to run. What's missed here is the scan may have spent upwards of a few seconds in the IPC queue before being serviced. In this case, the time limit may extend beyond the timeout of the request and the server will not return in time.

We should calculate the time limit from ServerCall.getReceiveTime instead to avoid these timeouts.

Issue Links

• links to: GitHub Pull Request #4562