[IMPALA-4275] Coordinator::GetNext() should not call WaitForAllInstances() at eos

• Type: Sub-task
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: Impala 2.8.0
• Component/s: Backend
• Target Version: Product Backlog
• Assignee: Unassigned
• Reporter: Henry Robinson
• Created: 11/Oct/16 19:44
• Updated: 30/Apr/20 22:16

Description

Coordinator::GetNext() calls WaitForAllInstances() (previously WaitForAllBackends()) when it finishes pulling the last batch. This puts fragment instance lifecycle management on the critical path to retrieve results, which doesn't make a lot of sense - the client should get notified that the query has returned all results without having to wait for every fragment instance to finish.

We should move that call to TearDown() instead. The reason that this is a little tricky is because TearDown() currently happens after the containing QueryExecState has been removed from the Impala server's exec state map. Once that's happened, fragment status reports can't get to the Coordinator, so WaitForAllInstances() would wait forever.

This JIRA is to fix both issues at once - calling TearDown() before deregistration to ensure fragments may be waited for, and to remove waiting from the GetNext() path.

Issue Links

• blocks: IMPALA-4333 Crash in PlanRootSink::GetNext()
• relates to: IMPALA-9380 Serialize query profile asynchronously