[IMPALA-6153] Prevent Coordinator::UpdateFilter() running after query exec resources are released

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 3.1.0
• Component/s: Backend
• Labels: query-lifecycle runtime-filters
• Epic Color: ghx-label-7
• Assignee: Pooja Nilangekar
• Reporter: Sailesh Mukil
• Created: 03/Nov/17 20:12
• Updated: 09/Aug/18 21:50
• Resolved: 09/Aug/18 21:50

Description

Coordinator::UpdateFilter() and CoordinatorBackendState::PublishFilter() run independent of the lifecycle of any fragment instance. This is problematic during query teardown.

Specifically we should not release resources for a query if any one of those above functions are still running for that query and we also should not not start running the above methods after resources are released for the query. Also, the 'rpc_params' in UpdateFilter() could potentially hold large amounts of untracked memory, so we should track it.

Issue Links

• is depended upon by: IMPALA-6412 Memory issues with processing of incoming global runtime filters on coordinator
• is related to: IMPALA-1575 Cancelled queries do not yield resources until close
• is related to: IMPALA-6144 Coordinator threads that publish RuntimeFilters continue to run after query failure/cancellation
• relates to: IMPALA-1575 Cancelled queries do not yield resources until close