[FLINK-23023] Support offset in window TVF - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Sub-task
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 1.14.0
Fix Version/s: 1.14.0
Component/s: Table SQL / Planner
Labels:
- pull-request-available

Description

Window offset is an optional parameter which could be used to change the alignment of windows.

There are something we need clarify about window offset:

(1) In SQL, window offset is an optional parameter, if it is specified, it is the last parameter of the window.

for Tumble window

TUMBLE(TABLE MyTable, DESCRIPTOR(rowtime), INTERVAL '15' MINUTE, INTERVAL '5' MINUTE)

for Hop Window

HOP(TABLE MyTable, DESCRIPTOR(rowtime), INTERVAL '1' MINUTE, INTERVAL '15' MINUTE,
INTERVAL '5' MINUTE)

for Cumulate Window

CUMULATE(TABLE MyTable, DESCRIPTOR(rowtime), INTERVAL '1' MINUTE, INTERVAL '15' MINUTE, INTERVAL '5' MINUTE)

(2) Window offset could be positive duration and negative duration.

(3) Window offset is used to change the alignment of Windows. The same record may be assigned to the different window after set window offset. But it always apply a rule, timestamp >= window_start && timestamp < window_end.

Give a demo, for a tumble window, window size is 10 MINUTE, which window would be assigned to for a record with timestamp 2021-06-30 00:00:04?

offset is '-16 MINUTE', the record assigns to window [2021-06-29 23:54:00, 2021-06-30 00:04:00)
offset is '-6 MINUTE', the record assigns to window [2021-06-29 23:54:00, 2021-06-30 00:04:00)
offset is '-4 MINUTE', the record assigns to window [2021-06-29 23:56:00, 2021-06-30 00:06:00)
offset is '0', the record assigns to window [2021-06-30 00:00:00, 2021-06-30 00:10:00)
offset is '4 MINUTE', the record assigns to window [2021-06-29 23:54:00, 2021-06-30 00:04:00)
offset is '6 MINUTE, the record assigns to window [2021-06-29 23:56:00, 2021-06-30 00:06:00)
offset is '16 MINUTE', the record assigns to window [2021-06-29 23:56:00, 2021-06-30 00:06:00)

(4) We could find that, some window offset parameters may have same effect on the alignment of windows, in the above case, '-16 MINUTE' /'-6 MINUTE'/'4 MINUTE' have same effect on a tumble window with '10 MINUTE' size.

(5) Window offset is only used to change the alignment of Windows, it has no effect on watermark.