Details
-
Bug
-
Status: Closed
-
Critical
-
Resolution: Fixed
-
1.3.2, 1.4.2, 1.5.0
-
None
Description
The built-in aggregation functions VAR_POP, VAR_SAMP, STDEV_POP, STDEV_SAMP are translated into regular AVG functions if they are applied in the context of a Group Window aggregation (GROUP BY TUMBLE/HOP/SESSION).
The reason is that these functions are internally represented as SqlAvgAggFunction but with different SqlKind. When translating Calcite aggregation functions to Flink Table agg functions, we only look at the type of the class, not at the value of the kind field. We did not notice that before, because in all other cases (regular GROUP BY without windows or OVER windows, we have a translation rule AggregateReduceFunctionsRule that decomposes the more complex functions into expressions of COUNT and SUM functions such that we never execute an AVG Flink function. That rule can only be applied on LogicalAggregate, however, we represent group windows as LogicalWindowAggregate, so the rule does not match.
We should fix this by:
1. restrict the translation to Flink avg functions in AggregateUtil to SqlKind.AVG.
2. implement a rule (hopefully based on AggregateReduceFunctionsRule) that decomposes the complex agg functions into the SUM and COUNT.
Step 1. is easy and a quick fix but we would get an exception "Unsupported Function" if VAR_POP is used in a GROUP BY window.
Step 2. might be more involved, depending on how difficult it is to port the rule.
Attachments
Attachments
Issue Links
- links to
