Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-5601

Window operator does not checkpoint watermarks

    XMLWordPrintableJSON

    Details

      Description

      During release testing Stefan Richter and I noticed that watermarks are not checkpointed in the window operator.

      This can lead to non determinism when restoring checkpoints. I was running an adjusted SessionWindowITCase via Kafka for testing migration and rescaling and ran into failures, because the data generator required determinisitic behaviour.

      What happened was that on restore it could happen that late elements were not dropped, because the watermarks needed to be re-established after restore first.

      Aljoscha Krettek Do you know whether there is a special reason for explicitly not checkpointing watermarks?

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                wind_ljy Jiayi Liao
                Reporter:
                uce Ufuk Celebi
              • Votes:
                0 Vote for this issue
                Watchers:
                20 Start watching this issue

                Dates

                • Created:
                  Updated: