[FLINK-8467] Restoring job that does not use checkpointing from savepoint breaks

• Type: Bug
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Component/s: Runtime / Checkpointing
• Labels: auto-deprioritized-major auto-deprioritized-minor
• Assignee: Unassigned
• Reporter: Jelmer Kuperus
• Created: 20/Jan/18 13:35
• Updated: 09/Dec/21 22:38

Description

When checkpointing is disabled, the Flink Kafka Consumer relies on the periodic offsets that are committed to the broker by the internal Kafka client.  Such a job would, upon restart, continue from the committed offsets.  However, in the situation that the job is restored from a savepoint, then the offsets within the savepoint supercede the broker-based offsets.

Here's a simple project that demonstrates the problem : https://github.com/jelmerk/flink-cancel-restart-job-without-checkpointing

And a link to the mailing list thread : http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Starting-a-job-that-does-not-use-checkpointing-from-a-savepoint-is-broken-td17874.html

 

If this is not something you want to address at least the upgrading guide (https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/upgrading.html)  should caution against this