[HIVE-21823] New metrics to get the average queue length / free executor number for a given time window

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Component/s: llap
• Labels: pull-request-available
• Assignee: Peter Vary
• Reporter: Peter Vary
• Created: 03/Jun/19 10:08
• Updated: 17/Nov/22 08:50
• Resolved: 16/Jun/19 21:51

Description

We need to calculate the average queue size / free executor size for a window to have good data for making routing decisions.

Interesting things to consider:

• The time gap between arriving request can be different, so simple average is not enough to have correct data
• We need to have 2 parameters
  • Time window length
  • Maximum data point numbers - so we will not collect "infinite" amount of data at high load

Issue Links

• links to: GitHub Pull Request #660