[IMPALA-6437] increase frequency of admission control topic updates

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 3.0, Impala 2.12.0
• Component/s: Distributed Exec
• Labels: admission-control resource-management
• Target Version: Impala 2.12.0
• Epic Color: ghx-label-3
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 24/Jan/18 17:19
• Updated: 03/Apr/18 20:09
• Resolved: 14/Feb/18 23:22

Description

Currently admission controls don't have a globally consistent view of how many queries have admitted so different coordinators can over-admit if queries arrive at multiple coordinators in-between topic updates. Topic updates happen every 2s by default. Worse, admission control topic updates can get stuck behind large catalog updates.

IMPALA-3649 proposes having a globally consistent view of admission control, which will require architectural changes.

In the meantime we could reduce the problem by increasing the frequency of topic updates for admission control and separating them from the larger catalog updates.

Issue Links

• is related to: IMPALA-6583 Various tests fail with missing database or table from catalog
• is related to: IMPALA-6785 Starting an Impalad on an already running cluster may result in inconsistent cluster subscription
• is related to: IMPALA-4140 Increase default threadpool size for statestore heartbeat and topic update
• relates to: IMPALA-3649 Globally consistent admission control