[FLINK-18934] Idle stream does not advance watermark in connected stream

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.11.0, 1.12.0, 1.13.0
• Fix Version/s: 1.14.0
• Component/s: API / DataStream, Runtime / Task
• Labels: pull-request-available stale-assigned
• Release Note: We added `processWatermarkStatusX` method to classes such as `AbstractStreamOperator`, `Input` etc. It allows to take the `WatermarkStatus` into account when combining watermarks in two/multi input operators.
• Assignee: Dawid Wysakowicz
• Reporter: Truong Duc Kien
• Created: 13/Aug/20 07:30
• Updated: 11/Oct/21 08:58
• Resolved: 08/Oct/21 15:15

Description

Per Flink documents, when a stream is idle, it will allow watermarks of downstream operator to advance. However, when I connect an active data stream with an idle data stream, the output watermark of the CoProcessOperator does not increase.

Here's a small test that reproduces the problem.

https://github.com/kien-truong/flink-idleness-testing

Issue Links

• relates to: FLINK-22904 Performance regression on 25.05.2021 in mapRebalanceMapSink
• relates to: FLINK-22881 Tasks are blocked while broadcasting stream status
• links to: GitHub Pull Request #15771
• links to: user@ mailing list thread