[FLINK-19079] Support row time deduplicate operator

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.12.0
• Component/s: Table SQL / Runtime
• Labels: pull-request-available
• Assignee: Leonard Xu
• Reporter: Leonard Xu
• Created: 28/Aug/20 03:42
• Updated: 05/Nov/20 13:17
• Resolved: 05/Nov/20 13:17

Description

To convert a insert-only table to versioned table, the recommended way is use deduplicate query as following, the converted versioned_view owns primary key and event time and thus can be a versioned table.

CREATE VIEW versioned_rates AS
SELECT currency, rate, currency_time
FROM (
      SELECT *,
      ROW_NUMBER() OVER (PARTITION BY currency   -- inferred primary key
            ORDER BY currency_time               -- the event time  
        DESC) AS rowNum
      FROM rates)
WHERE rowNum = 1;

But currently deduplicate operator only support on process time, this issue aims to support deduplicate on Event time.

 

Issue Links

• links to: GitHub Pull Request #13331