[FLINK-23023] Support offset in window TVF

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.14.0
• Fix Version/s: 1.14.0
• Component/s: Table SQL / Planner
• Labels: pull-request-available
• Assignee: Jing Zhang
• Reporter: Jing Zhang
• Created: 17/Jun/21 13:01
• Updated: 23/Sep/21 18:01
• Resolved: 12/Jul/21 03:06

Description

Window offset is an optional parameter which could be used to change the alignment of windows.

There are something we need clarify about window offset:

(1) In SQL, window offset is an optional parameter, if it is specified, it is the last parameter of the window.

for Tumble window

TUMBLE(TABLE MyTable, DESCRIPTOR(rowtime), INTERVAL '15' MINUTE, INTERVAL '5' MINUTE)

for Hop Window

HOP(TABLE MyTable, DESCRIPTOR(rowtime), INTERVAL '1' MINUTE, INTERVAL '15' MINUTE,
INTERVAL '5' MINUTE)

for Cumulate Window

CUMULATE(TABLE MyTable, DESCRIPTOR(rowtime), INTERVAL '1' MINUTE, INTERVAL '15' MINUTE, INTERVAL '5' MINUTE)

(2) Window offset could be positive duration and negative duration.

(3) Window offset is used to change the alignment of Windows. The same record may be assigned to the different window after set window offset. But it always apply a rule, timestamp >= window_start && timestamp < window_end.

Give a demo, for a tumble window, window size is 10 MINUTE, which window would be assigned to for a record with timestamp 2021-06-30 00:00:04?

1. offset is '-16 MINUTE',  the record assigns to window [2021-06-29 23:54:00, 2021-06-30 00:04:00)
2. offset is '-6 MINUTE', the record assigns to window [2021-06-29 23:54:00, 2021-06-30 00:04:00)
3. offset is '-4 MINUTE', the record assigns to window [2021-06-29 23:56:00, 2021-06-30 00:06:00)
4. offset is '0', the record assigns to window [2021-06-30 00:00:00, 2021-06-30 00:10:00)
5. offset is '4 MINUTE', the record assigns to window [2021-06-29 23:54:00, 2021-06-30 00:04:00)
6. offset is '6 MINUTE, the record assigns to window [2021-06-29 23:56:00, 2021-06-30 00:06:00)
7. offset is '16 MINUTE', the record assigns to window [2021-06-29 23:56:00, 2021-06-30 00:06:00)

(4) We could find that, some window offset parameters may have same effect on the alignment of windows, in the above case,  '-16 MINUTE' /'-6 MINUTE'/'4 MINUTE' have same effect on a tumble window with '10 MINUTE'  size.

(5) Window offset is only used to change the alignment of Windows, it has no effect on watermark.

Issue Links

• is duplicated by: FLINK-17767 Add offset support for TUMBLE, HOP, SESSION group windows
• relates to: FLINK-23055 Add document for Window TVF offset
• relates to: FLINK-17767 Add offset support for TUMBLE, HOP, SESSION group windows
• links to: GitHub Pull Request #16215