[FLINK-7728] StatusWatermarkValve has different min watermark advancement behavior depending on the ordering inputs become idle

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.2.1, 1.3.2, 1.4.0
• Fix Version/s: 1.3.4, 1.4.0
• Component/s: API / DataStream
• Assignee: Tzu-Li (Gordon) Tai
• Reporter: Tzu-Li (Gordon) Tai
• Created: 28/Sep/17 12:34
• Updated: 12/Mar/18 15:24
• Resolved: 12/Mar/18 15:24

Description

Currently, once all inputs of a StatusWatermarkValve becomes idle, we only emit the StreamStatus.IDLE marker, and check nothing else. This makes the watermark advancement behavior inconsistent in the case that all inputs become idle, depending on the order that they become idle.

Consider the following setup:

Channel #1: Watermark 10, ACTIVE
Channel #2: Watermark 5, ACTIVE
Channel #3: Watermark 5, ACTIVE

If the channels become IDLE with the order #2 -> #3 -> #1, watermark 10 will be emitted as the new min watermark once both #2 and #3 becomes idle.
On the other hand, if the order is #1 -> #2 -> #3, watermark 10 will not be emitted.

Issue Links

• links to: GitHub Pull Request #4747