[FLINK-5601] Window operator does not checkpoint watermarks

• Type: Improvement
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Affects Version/s: 1.5.0, 1.6.0, 1.7.0, 1.8.0, 1.9.0, 1.10.0, 1.11.0
• Component/s: Runtime / Checkpointing
• Labels: auto-deprioritized-critical auto-deprioritized-major auto-deprioritized-minor auto-unassigned pull-request-available
• Assignee: Unassigned
• Reporter: Ufuk Celebi
• Created: 21/Jan/17 16:41
• Updated: 01/Oct/24 02:58

Description

During release testing [e-mail] and I noticed that watermarks are not checkpointed in the window operator.

This can lead to non determinism when restoring checkpoints. I was running an adjusted SessionWindowITCase via Kafka for testing migration and rescaling and ran into failures, because the data generator required determinisitic behaviour.

What happened was that on restore it could happen that late elements were not dropped, because the watermarks needed to be re-established after restore first.

aljoscha Do you know whether there is a special reason for explicitly not checkpointing watermarks?

Issue Links

• causes: FLINK-18053 Savepoints do not preserve watermarks
• links to: GitHub Pull Request #7013