[FLINK-19109] Split Reader eats chained periodic watermarks - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Blocker
Resolution: Fixed
Affects Version/s: 1.10.0, 1.10.1, 1.10.2, 1.11.0, 1.11.1
Fix Version/s: 1.10.3, 1.11.2, 1.12.0
Component/s: Runtime / Task
Labels:
- pull-request-available

Description

Attempting to generate watermarks chained to the Split Reader / ContinuousFileReaderOperator, as in

SingleOutputStreamOperator<Event> results = env
  .readTextFile(...)
  .map(...)
  .assignTimestampsAndWatermarks(bounded)
  .keyBy(...)
  .process(...);

leads to the Watermarks failing to be produced. Breaking the chain, via disableOperatorChaining() or a rebalance, works around the bug. Using punctuated watermarks also avoids the issue.

Looking at this in the debugger reveals that timer service is being prematurely quiesced.

In many respects this is ~~FLINK-7666~~ brought back to life.

The problem is not present in 1.9.3.

There's a minimal reproducible example in https://github.com/alpinegizmo/flink-question-001/tree/bug.