[IMPALA-7829] Send the final profile only after all fragment instances have been closed

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 3.2.0
• Fix Version/s: Impala 3.2.0
• Component/s: Distributed Exec
• Target Version: Impala 3.2.0
• Epic Color: ghx-label-3
• Assignee: Michael Ho
• Reporter: Michael Ho
• Created: 07/Nov/18 18:21
• Updated: 26/Nov/18 18:35
• Resolved: 26/Nov/18 18:35

Description

As shown in IMPALA-7828. there is some non-determinism on whether the errors detected in FragmentInstanceState::Close() will show up in the final profile sent to the coordinator. The reason is that the current code marks a fragment instance "done" after ExecInternal() completes but before Close() is called. There is a window in between when the final status report may be sent before Close() finishes. We should consider not sending the final report until Close() is done but we probably need to understand whether it has any implication to the overall reported query execution time. It should have no effect to the "First Rows Available" time for a query.