[IMPALA-9151] Number of executors during planning needs to account for suspended executor groupa

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 3.4.0
• Component/s: Frontend
• Target Version: Impala 3.4.0
• Epic Color: ghx-label-2
• Assignee: Lars Volker
• Reporter: Lars Volker
• Created: 12/Nov/19 18:40
• Updated: 04/Dec/19 17:32
• Resolved: 04/Dec/19 00:46

Description

When configuring Impala with executor groups, the planner might see a ExecutorMembershipSnapshot that has no executors in it. This can happen if the first executor group has not started up yet, or if all executor groups have been shutdown. If this happens, the planner will make sub-optimal decisions, e.g. decide on a broadcast join vs a PHJ. In the former case, we should have a configurable fallback cluster size to use during planning. In the latter case, we should hang on to the last executor group size that we had observed.