[FLINK-23747] Testing Window TVF offset

• Type: Improvement
• Status: Closed
• Priority: Blocker
• Resolution: Done
• Fix Version/s: 1.14.0
• Component/s: Tests
• Labels: release-testing
• Assignee: liwei li
• Reporter: Jing Zhang
• Created: 13/Aug/21 03:11
• Updated: 30/Sep/21 11:52
• Resolved: 10/Sep/21 02:40

Description

Window offset is an optional parameter which could be used to change the alignment of windows.

There are something we need clarify about window offset:

(1) In SQL, window offset is an optional parameter, if it is specified, it is the last parameter of the window.

for Tumble window

TUMBLE(TABLE MyTable, DESCRIPTOR(rowtime), INTERVAL '15' MINUTE, INTERVAL '5' MINUTE)

for Hop Window

HOP(TABLE MyTable, DESCRIPTOR(rowtime), INTERVAL '1' MINUTE, INTERVAL '15' MINUTE,
INTERVAL '5' MINUTE)

for Cumulate Window

CUMULATE(TABLE MyTable, DESCRIPTOR(rowtime), INTERVAL '1' MINUTE, INTERVAL '15' MINUTE, INTERVAL '5' MINUTE)

(2) Window offset could be positive duration and negative duration.

(3) Window offset is used to change the alignment of Windows. The same record may be assigned to the different window after set window offset. But it always apply a rule, timestamp >= window_start && timestamp < window_end.

Give a demo, for a tumble window, window size is 10 MINUTE, which window would be assigned to for a record with timestamp 2021-06-30 00:00:04?

1. offset is '-16 MINUTE',  the record assigns to window [2021-06-29 23:54:00, 2021-06-30 00:04:00)
2. offset is '-6 MINUTE', the record assigns to window [2021-06-29 23:54:00, 2021-06-30 00:04:00)
3. offset is '-4 MINUTE', the record assigns to window [2021-06-29 23:56:00, 2021-06-30 00:06:00)
4. offset is '0', the record assigns to window [2021-06-30 00:00:00, 2021-06-30 00:10:00)
5. offset is '4 MINUTE', the record assigns to window [2021-06-29 23:54:00, 2021-06-30 00:04:00)
6. offset is '6 MINUTE, the record assigns to window [2021-06-29 23:56:00, 2021-06-30 00:06:00)
7. offset is '16 MINUTE', the record assigns to window [2021-06-29 23:56:00, 2021-06-30 00:06:00)

(4) We could find that, some window offset parameters may have same effect on the alignment of windows, in the above case,  '-16 MINUTE' /'-6 MINUTE'/'4 MINUTE' have same effect on a tumble window with '10 MINUTE'  size.

(5) Window offset is only used to change the alignment of Windows, it has no effect on watermark.