[IMPALA-5486] Port control-plane parts of ImpalaInternalService to KRPC

• Type: Task
• Status: Reopened
• Priority: Major
• Resolution: Unresolved
• Component/s: Distributed Exec
• Epic Link: Impala Scalability Improvement
• Epic Color: ghx-label-2
• Assignee: Wenzhe Zhou
• Reporter: Henry Robinson
• Created: 12/Jun/17 18:24
• Updated: 23/Dec/20 18:11

Issue Links

• depends upon: IMPALA-7213 Port ReportExecStatus() RPCs to KRPC
• is depended upon by: IMPALA-6412 Memory issues with processing of incoming global runtime filters on coordinator
• relates to: IMPALA-5225 Queries that reference >6Million files and 500K partitions can fail with java.lang.OutOfMemoryError while serializing PlanFragment

Sub-Tasks

• Abort ExecFInstance() RPC loop early after query failure: Resolved
• Don't make RPCs from Coordinator::UpdateBackendExecStatus(): Open
• On large secure clusters the connection setup thread becomes bottleneck at warmup and cause occasional timeout failures: Resolved
• Make Coordinator ExecState an atomic enum: Resolved
• Respond to ReportExecStatus() RPC with CANCELLED whenever query execution has terminated: Resolved
• Add TCP timeouts to all RPCs that don't block: Resolved
• Port ExecQueryFInstances() to KPRC: Resolved
• Port CancelQueryFInstances() to KRPC: Resolved
• Coordinator should cancel backends when returning EOS: Reopened
• Port RemoteShutdown() to KRPC: Resolved
• Convert ExecQueryFInstance() RPC to become asynchronous: Resolved