[IMPALA-9711] Incrementally compute averaged profile

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.0.0
• Target Version: Impala 4.0.0
• Epic Color: ghx-label-2
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 30/Apr/20 05:41
• Updated: 17/Nov/20 18:47
• Resolved: 17/Nov/20 18:47

Description

The parent showed that updating the averaged profile can take a bit of time and executes on the RPC handler of the fetch RPC. We should consider how to optimise this.

I think the most impactful change, from the point of view of query latency, is to apply the profile updates when each instance finishes instead of doing the computation on the RPC handler.

For IMPALA-9382, we also need the averaged profile to be kept up-to-date when the profile is accessed, because it will be the only part of the profile that is exposed to clients.

Issue Links

• breaks: IMPALA-10276 Release build sees SIGSEGV when updating the total time counter
• relates to: IMPALA-9756 Queries are not guaranteed to be cancelled before unregistration