Details
-
Improvement
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
Impala 4.3.0
-
None
-
ghx-label-6
Description
In multiple executor group set setup, Frontend will try to match a query with the smallest executor group set that can fit the memory and cpu requirement of the compiled query. There are kind of query where the compiled plan will fit to any executor group set but not necessarily deliver the best performance. An example for this is Impala's COMPUTE STATS query. It does full table scan and aggregate the stats, have fairly simple query plan shape, but can benefit from higher scan parallelism.
Planner needs to give additional feedback to Frontend that the query might be under parallelized under current executor group. Frontend can then make judgement whether to assign the compiled plan to current executor group anyway, or try step up to the next larger executor group and increase parallelism.