[FLINK-9422] Dedicated DISTINCT operator for streaming tables with time attributes

• Type: Improvement
• Status: Reopened
• Priority: Not a Priority
• Resolution: Unresolved
• Component/s: Table SQL / Legacy Planner
• Labels: stale-assigned
• Assignee: Ruidong Li
• Reporter: Fabian Hueske
• Created: 23/May/18 10:05
• Updated: 29/Jul/21 07:35

Description

We can implement a dedicated operator for a UNION operator on tables with time attributes. Currently, UNION is translated into a UNION ALL and a subsequent GROUP BY on all attributes without aggregation functions. The state of the grouping operator is only clean up using state retention timers.

The dedicated operator would leverage the monotonicity property of the time attribute and watermarks to automatically clean up its state.

Issue Links

• relates to: FLINK-8594 Add a dedicated DISTINCT operator to always produce append sreams