[FLINK-17713] "Stateful stream job upgrade end-to-end test" fails

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Component/s: Runtime / Checkpointing
• Assignee: Unassigned
• Reporter: Robert Metzger
• Created: 15/May/20 05:36
• Updated: 15/May/20 05:45
• Resolved: 15/May/20 05:45

Description

CI: https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=1348&view=logs&j=c88eea3b-64a0-564d-0031-9fdcd7b8abee&t=1e2bbe5b-4657-50be-1f07-d84bfce5b1f5

Potentially caused by this:

2020-05-15T04:46:20.7037837Z 2020-05-15 04:46:11,134 WARN  org.apache.flink.runtime.checkpoint.StandaloneCompletedCheckpointStore [] - Fail to subsume the old checkpoint.
2020-05-15T04:46:20.7038858Z java.io.IOException: Directory /home/vsts/work/1/s/flink-end-to-end-tests/test-scripts/temp-test-directory-56851042201/savepoint-e2e-test-chckpt-dir/33e34191d2f3c84d9b7eb5898d3a34fc/chk-3 is not empty
2020-05-15T04:46:20.7039955Z 	at org.apache.flink.core.fs.local.LocalFileSystem.delete(LocalFileSystem.java:192) ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
2020-05-15T04:46:20.7041042Z 	at org.apache.flink.runtime.state.filesystem.FsCompletedCheckpointStorageLocation.disposeStorageLocation(FsCompletedCheckpointStorageLocation.java:70) ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
2020-05-15T04:46:20.7042317Z 	at org.apache.flink.runtime.checkpoint.CompletedCheckpoint.doDiscard(CompletedCheckpoint.java:264) ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
2020-05-15T04:46:20.7043438Z 	at org.apache.flink.runtime.checkpoint.CompletedCheckpoint.discardOnSubsume(CompletedCheckpoint.java:219) ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
2020-05-15T04:46:20.7044540Z 	at org.apache.flink.runtime.checkpoint.StandaloneCompletedCheckpointStore.addCheckpoint(StandaloneCompletedCheckpointStore.java:72) ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
2020-05-15T04:46:20.7045684Z 	at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.completePendingCheckpoint(CheckpointCoordinator.java:1003) ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
2020-05-15T04:46:20.7047005Z 	at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.receiveAcknowledgeMessage(CheckpointCoordinator.java:910) ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
2020-05-15T04:46:20.7048039Z 	at org.apache.flink.runtime.scheduler.SchedulerBase.lambda$acknowledgeCheckpoint$4(SchedulerBase.java:802) ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
2020-05-15T04:46:20.7048663Z 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_252]
2020-05-15T04:46:20.7049114Z 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_252]
2020-05-15T04:46:20.7049643Z 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_252]
2020-05-15T04:46:20.7050291Z 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.8.0_252]
2020-05-15T04:46:20.7057390Z 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_252]
2020-05-15T04:46:20.7058064Z 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_252]
2020-05-15T04:46:20.7058510Z 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]
2

Issue Links

• is caused by: FLINK-17467 Implement aligned savepoint in UC mode