[IMPALA-9155] Single Admission Controller per Cluster

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: Impala 3.4.0
• Component/s: Backend
• Epic Color: ghx-label-10
• Assignee: Bikramjeet Vig
• Reporter: Lars Volker
• Created: 13/Nov/19 21:05
• Updated: 02/Apr/21 17:06

Description

We should consider building a single admission control service that can admit queries for multiple coordinators. This would simplify the current approach where coordinators make distributed decisions, remove the risk of overadmission by concurrent decisions, and allow us to implement more sophisticated admission control strategies.

Issue Links

• relates to: IMPALA-3649 Globally consistent admission control

Sub-Tasks

• Model QuerySchedule as a protobuf: Resolved
• Introduce new admission control rpc service: Resolved
• Introduce new admission control daemon: Resolved
• Implement recovery for the admission control service: Open
• Evaluate AdmitQuery params for efficiency: Open
• Eliminate blocking in rpc threads in AdmissionControlService: Open
• Reconsider admission control heuristics with admissiond: Open
• Add retrying of AdmitQuery: Resolved
• Ensure admissiond stays in sync with coordinators: Resolved
• Fix issues with failed ReleaseQueryBackends rpc: Resolved
• Handle failed coordinators in admissiond: Resolved
• Add versioning to admission heartbeats: Resolved
• Fix handling of queued queries for coordinator failure modes and during cancellation: Open
• Ensure consistency between failure detection and registration/Ack of a coordinator by the admission service: Open