[FLINK-5972] Don't allow shrinking merging windows

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.1.0, 1.2.0, 1.3.0
• Fix Version/s: 1.2.1, 1.3.0
• Component/s: API / DataStream
• Assignee: Aljoscha Krettek
• Reporter: Aljoscha Krettek
• Created: 06/Mar/17 16:08
• Updated: 24/Mar/17 07:10
• Resolved: 24/Mar/17 07:10

Description

A misbehaving MergingWindowAssigner can cause a merge that results in a window that is smaller than the span of all the merged windows. This, in itself is not problematic. It becomes problematic when the end timestamp of a window that was not late before merging is now earlier than the watermark (the timestamp is smaller than the watermark).

There are two choices:

• immediately process the window
• drop the window

processing the window will lead to late data downstream.

The current behaviour is to silently drop the window but that logic has a bug: we only remove the dropped window from the MergingWindowSet but we don't properly clean up state and timers that the window still (possibly) has. We should fix this bug in the process of resolving this issue.

We should either just fix the bug and still silently drop windows or add a check and throw an exception when the end timestamp falls below the watermark.

Issue Links

• links to: GitHub Pull Request #3587