Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-19109

Split Reader eats chained periodic watermarks

    XMLWordPrintableJSON

    Details

      Description

      Attempting to generate watermarks chained to the Split Reader / ContinuousFileReaderOperator, as in

      SingleOutputStreamOperator<Event> results = env
        .readTextFile(...)
        .map(...)
        .assignTimestampsAndWatermarks(bounded)
        .keyBy(...)
        .process(...);

      leads to the Watermarks failing to be produced. Breaking the chain, via disableOperatorChaining() or a rebalance, works around the bug. Using punctuated watermarks also avoids the issue.

      Looking at this in the debugger reveals that timer service is being prematurely quiesced.

      In many respects this is FLINK-7666 brought back to life.

      The problem is not present in 1.9.3.

      There's a minimal reproducible example in https://github.com/alpinegizmo/flink-question-001/tree/bug.

        Attachments

          Activity

            People

            • Assignee:
              roman_khachatryan Roman Khachatryan
              Reporter:
              alpinegizmo David Anderson
            • Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: