[FLINK-23771] FLIP-147 Checkpoint N has not been started at all

• Type: Sub-task
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.14.0
• Fix Version/s: 1.14.0
• Component/s: Runtime / Checkpointing
• Labels: pull-request-available
• Assignee: Yun Gao
• Reporter: Piotr Nowojski
• Created: 13/Aug/21 18:35
• Updated: 16/Aug/21 07:48
• Resolved: 16/Aug/21 07:48

Description

Observed when running WIP version of https://github.com/apache/flink/pull/16773/

6311 [flink-akka.actor.default-dispatcher-6] INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph [] - transform-2-keyed (4/4) (7f9e36a73792074afd7803e39f563fac) switched from RUNNING to FAILED on dac2cc5f-41f8-4067-99a9-ae56341e3735 @ localhost (dataPort=-1).
java.lang.Exception: Could not perform checkpoint 4 for operator transform-2-keyed (4/4)#1.
	at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointAsyncInMailbox(StreamTask.java:1170) ~[classes/:?]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$triggerCheckpointAsync$11(StreamTask.java:1117) ~[classes/:?]
	at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[classes/:?]
	at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90) ~[classes/:?]
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsWhenDefaultActionUnavailable(MailboxProcessor.java:338) ~[classes/:?]
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:324) ~[classes/:?]
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:201) ~[classes/:?]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.afterInvoke(StreamTask.java:851) ~[classes/:?]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.executeInvoke(StreamTask.java:754) ~[classes/:?]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.runWithCleanUpOnFail(StreamTask.java:787) ~[classes/:?]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:730) ~[classes/:?]
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:786) ~[classes/:?]
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:572) ~[classes/:?]
	at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_131]
Caused by: java.lang.IllegalStateException: Checkpoint 4 has not been started at all
	at org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.getAllBarriersReceivedFuture(SingleCheckpointBarrierHandler.java:464) ~[classes/:?]
	at org.apache.flink.streaming.runtime.io.checkpointing.CheckpointedInputGate.getAllBarriersReceivedFuture(CheckpointedInputGate.java:209) ~[classes/:?]
	at org.apache.flink.streaming.runtime.io.StreamTaskNetworkInput.prepareSnapshot(StreamTaskNetworkInput.java:112) ~[classes/:?]
	at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.prepareSnapshot(StreamOneInputProcessor.java:83) ~[classes/:?]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.prepareInputSnapshot(StreamTask.java:458) ~[classes/:?]
	at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.prepareInflightDataSnapshot(SubtaskCheckpointCoordinatorImpl.java:554) ~[classes/:?]
	at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.initInputsCheckpoint(SubtaskCheckpointCoordinatorImpl.java:423) ~[classes/:?]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointAsyncInMailbox(StreamTask.java:1154) ~[classes/:?]
	... 13 more

Issue Links

• links to: GitHub Pull Request #16827