[IMPALA-1514] DataStreamSender (and possible Coordinator) has too many sender threads

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: Impala 2.0
• Fix Version/s: Product Backlog
• Component/s: Distributed Exec
• Labels: performance
• Target Version: Product Backlog
• Assignee: Henry Robinson
• Reporter: Alan Choi
• Created: 21/Nov/14 23:47
• Updated: 20/Jun/17 00:46
• Resolved: 20/Jun/17 00:46

Description

DataStreamSender creates one thread per EXCHANGE destination per query. On a large cluster with a highly concurrent workload, this will create too many threads. The immediate impact is that the thread creation time is dominating the query execution time (i.e. the prepare time is getting very high).