[FLINK-9419] UNION should not be treated as retraction producing operator

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Component/s: Table SQL / API
• Assignee: Hequn Cheng
• Reporter: Fabian Hueske
• Created: 23/May/18 10:00
• Updated: 23/May/18 16:05
• Resolved: 23/May/18 16:05

Description

The following query fails

SELECT
    user_id,
    count(msg),
    HOP_END(rowtime, INTERVAL '1' second, INTERVAL '1' minute)
FROM (SELECT rowtime, user_id, action_name AS msg FROM
          event_client_action
        WHERE /* various clauses */
        UNION SELECT rowtime, user_id, action_type AS msg FROM
           event_server_action
           WHERE /* various clauses */
      )
GROUP BY
HOP(rowtime, INTERVAL '1' second, INTERVAL '1' minute), user_id

with

Retraction on windowed GroupBy aggregation is not supported yet. Note: Windowed GroupBy aggregation should not follow a non-windowed GroupBy aggregation.

The problem is that the UNION operator is translated into a UNION ALL and a subsequent GROUP BY on all attributes without an aggregation function. Currently, all GROUP BY operators are treated as retraction-producing operators. However, this is only true for grouping operators with aggregation functions. If the operator groups on all attributes and has no aggregation functions, it does not produce retractions but only forwards them (similar to a filter operator).

Issue Links

• is blocked by: FLINK-8566 Replace retract/insert of same record for state retention timer resets