Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-26501

Quickstarts Scala nightly end-to-end test failed on azure due to checkponts failed and logs contains exceptions

    XMLWordPrintableJSON

Details

    Description

      2022-03-05T02:35:36.4040037Z Mar 05 02:35:36 2022-03-05 02:35:34,334 INFO  org.apache.flink.runtime.checkpoint.CheckpointCoordinator    [] - Triggering checkpoint 1 (type=CHECKPOINT) @ 1646447734295 for job b236087395260dc34648b84c2b86d6e8.
2022-03-05T02:35:36.4041701Z Mar 05 02:35:36 2022-03-05 02:35:34,387 INFO  org.apache.flink.runtime.checkpoint.CheckpointCoordinator    [] - Decline checkpoint 1 by task e8a324cae6bf452d32db6797bbbafad0 of job b236087395260dc34648b84c2b86d6e8 at 127.0.0.1:45911-0a50f5 @ localhost (dataPort=44047).
2022-03-05T02:35:36.4043279Z Mar 05 02:35:36 org.apache.flink.util.SerializedThrowable: Task name with subtask : Source: Sequence Source (Deprecated) -> Map -> Sink: Unnamed (1/1)#0 Failure reason: Checkpoint was declined (task is closing)
2022-03-05T02:35:36.4044531Z Mar 05 02:35:36 	at org.apache.flink.runtime.taskmanager.Task.declineCheckpoint(Task.java:1389) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4045729Z Mar 05 02:35:36 	at org.apache.flink.runtime.taskmanager.Task.declineCheckpoint(Task.java:1382) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4047172Z Mar 05 02:35:36 	at org.apache.flink.runtime.taskmanager.Task.triggerCheckpointBarrier(Task.java:1348) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4049092Z Mar 05 02:35:36 	at org.apache.flink.runtime.taskexecutor.TaskExecutor.triggerCheckpoint(TaskExecutor.java:956) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4050158Z Mar 05 02:35:36 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_322]
2022-03-05T02:35:36.4050929Z Mar 05 02:35:36 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_322]
2022-03-05T02:35:36.4051776Z Mar 05 02:35:36 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_322]
2022-03-05T02:35:36.4052559Z Mar 05 02:35:36 	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_322]
2022-03-05T02:35:36.4053373Z Mar 05 02:35:36 	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$handleRpcInvocation$1(AkkaRpcActor.java:316) ~[?:?]
2022-03-05T02:35:36.4054849Z Mar 05 02:35:36 	at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83) ~[?:?]
2022-03-05T02:35:36.4055685Z Mar 05 02:35:36 	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:314) ~[?:?]
2022-03-05T02:35:36.4056461Z Mar 05 02:35:36 	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:217) ~[?:?]
2022-03-05T02:35:36.4057219Z Mar 05 02:35:36 	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:163) ~[?:?]
2022-03-05T02:35:36.4057899Z Mar 05 02:35:36 	at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) ~[?:?]
2022-03-05T02:35:36.4059666Z Mar 05 02:35:36 	at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) ~[?:?]
2022-03-05T02:35:36.4061005Z Mar 05 02:35:36 	at scala.PartialFunction.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4062324Z Mar 05 02:35:36 	at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4063941Z Mar 05 02:35:36 	at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) ~[?:?]
2022-03-05T02:35:36.4065009Z Mar 05 02:35:36 	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4066205Z Mar 05 02:35:36 	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4067514Z Mar 05 02:35:36 	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4068255Z Mar 05 02:35:36 	at akka.actor.Actor.aroundReceive(Actor.scala:537) ~[?:?]
2022-03-05T02:35:36.4069019Z Mar 05 02:35:36 	at akka.actor.Actor.aroundReceive$(Actor.scala:535) ~[?:?]
2022-03-05T02:35:36.4069638Z Mar 05 02:35:36 	at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) ~[?:?]
2022-03-05T02:35:36.4070271Z Mar 05 02:35:36 	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:580) ~[?:?]
2022-03-05T02:35:36.4070862Z Mar 05 02:35:36 	at akka.actor.ActorCell.invoke(ActorCell.scala:548) ~[?:?]
2022-03-05T02:35:36.4071453Z Mar 05 02:35:36 	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) ~[?:?]
2022-03-05T02:35:36.4072430Z Mar 05 02:35:36 	at akka.dispatch.Mailbox.run(Mailbox.scala:231) ~[?:?]
2022-03-05T02:35:36.4073023Z Mar 05 02:35:36 	at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ~[?:?]
2022-03-05T02:35:36.4073687Z Mar 05 02:35:36 	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) ~[?:1.8.0_322]
2022-03-05T02:35:36.4074596Z Mar 05 02:35:36 	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) ~[?:1.8.0_322]
2022-03-05T02:35:36.4075712Z Mar 05 02:35:36 	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) ~[?:1.8.0_322]
2022-03-05T02:35:36.4076437Z Mar 05 02:35:36 	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) ~[?:1.8.0_322]
2022-03-05T02:35:36.4077754Z Mar 05 02:35:36 2022-03-05 02:35:34,410 WARN  org.apache.flink.runtime.checkpoint.CheckpointFailureManager [] - Failed to trigger checkpoint 1 for job b236087395260dc34648b84c2b86d6e8. (0 consecutive failed attempts so far)
2022-03-05T02:35:36.4078865Z Mar 05 02:35:36 org.apache.flink.runtime.checkpoint.CheckpointException: Checkpoint was declined (task is closing)
2022-03-05T02:35:36.4080161Z Mar 05 02:35:36 	at org.apache.flink.runtime.messages.checkpoint.SerializedCheckpointException.unwrap(SerializedCheckpointException.java:51) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4081619Z Mar 05 02:35:36 	at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.receiveDeclineMessage(CheckpointCoordinator.java:988) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4083063Z Mar 05 02:35:36 	at org.apache.flink.runtime.scheduler.ExecutionGraphHandler.lambda$declineCheckpoint$2(ExecutionGraphHandler.java:103) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4085407Z Mar 05 02:35:36 	at org.apache.flink.runtime.scheduler.ExecutionGraphHandler.lambda$processCheckpointCoordinatorMessage$3(ExecutionGraphHandler.java:119) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4086635Z Mar 05 02:35:36 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_322]
2022-03-05T02:35:36.4087419Z Mar 05 02:35:36 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_322]
2022-03-05T02:35:36.4088438Z Mar 05 02:35:36 	at java.lang.Thread.run(Thread.java:750) [?:1.8.0_322]
2022-03-05T02:35:36.4089614Z Mar 05 02:35:36 Caused by: org.apache.flink.util.SerializedThrowable: Task name with subtask : Source: Sequence Source (Deprecated) -> Map -> Sink: Unnamed (1/1)#0 Failure reason: Checkpoint was declined (task is closing)
2022-03-05T02:35:36.4090937Z Mar 05 02:35:36 	at org.apache.flink.runtime.taskmanager.Task.declineCheckpoint(Task.java:1389) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4092177Z Mar 05 02:35:36 	at org.apache.flink.runtime.taskmanager.Task.declineCheckpoint(Task.java:1382) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4093430Z Mar 05 02:35:36 	at org.apache.flink.runtime.taskmanager.Task.triggerCheckpointBarrier(Task.java:1348) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4094740Z Mar 05 02:35:36 	at org.apache.flink.runtime.taskexecutor.TaskExecutor.triggerCheckpoint(TaskExecutor.java:956) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4095836Z Mar 05 02:35:36 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_322]
2022-03-05T02:35:36.4096579Z Mar 05 02:35:36 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_322]
2022-03-05T02:35:36.4097766Z Mar 05 02:35:36 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_322]
2022-03-05T02:35:36.4098684Z Mar 05 02:35:36 	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_322]
2022-03-05T02:35:36.4101381Z Mar 05 02:35:36 	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$handleRpcInvocation$1(AkkaRpcActor.java:316) ~[?:?]
2022-03-05T02:35:36.4102353Z Mar 05 02:35:36 	at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83) ~[?:?]
2022-03-05T02:35:36.4103218Z Mar 05 02:35:36 	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:314) ~[?:?]
2022-03-05T02:35:36.4104019Z Mar 05 02:35:36 	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:217) ~[?:?]
2022-03-05T02:35:36.4104801Z Mar 05 02:35:36 	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:163) ~[?:?]
2022-03-05T02:35:36.4105719Z Mar 05 02:35:36 	at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) ~[?:?]
2022-03-05T02:35:36.4108356Z Mar 05 02:35:36 	at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) ~[?:?]
2022-03-05T02:35:36.4110333Z Mar 05 02:35:36 	at scala.PartialFunction.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4112523Z Mar 05 02:35:36 	at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4113601Z Mar 05 02:35:36 	at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) ~[?:?]
2022-03-05T02:35:36.4114790Z Mar 05 02:35:36 	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4116110Z Mar 05 02:35:36 	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4117636Z Mar 05 02:35:36 	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-03-05T02:35:36.4118641Z Mar 05 02:35:36 	at akka.actor.Actor.aroundReceive(Actor.scala:537) ~[?:?]
2022-03-05T02:35:36.4119307Z Mar 05 02:35:36 	at akka.actor.Actor.aroundReceive$(Actor.scala:535) ~[?:?]
2022-03-05T02:35:36.4120161Z Mar 05 02:35:36 	at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) ~[?:?]
2022-03-05T02:35:36.4120842Z Mar 05 02:35:36 	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:580) ~[?:?]
2022-03-05T02:35:36.4121482Z Mar 05 02:35:36 	at akka.actor.ActorCell.invoke(ActorCell.scala:548) ~[?:?]
2022-03-05T02:35:36.4122113Z Mar 05 02:35:36 	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) ~[?:?]
2022-03-05T02:35:36.4122736Z Mar 05 02:35:36 	at akka.dispatch.Mailbox.run(Mailbox.scala:231) ~[?:?]
2022-03-05T02:35:36.4123332Z Mar 05 02:35:36 	at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ~[?:?]
2022-03-05T02:35:36.4123984Z Mar 05 02:35:36 	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) ~[?:1.8.0_322]
2022-03-05T02:35:36.4124749Z Mar 05 02:35:36 	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) ~[?:1.8.0_322]
2022-03-05T02:35:36.4125750Z Mar 05 02:35:36 	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) ~[?:1.8.0_322]
2022-03-05T02:35:36.4126591Z Mar 05 02:35:36 	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) ~[?:1.8.0_322]
2022-03-05T02:35:36.4128133Z Mar 05 02:35:36 2022-03-05 02:35:34,430 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph       [] - Source: Sequence Source (Deprecated) -> Map -> Sink: Unnamed (1/1) (e8a324cae6bf452d32db6797bbbafad0) switched from RUNNING to FINISHED.

      https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=32553&view=logs&j=91bf6583-3fb2-592f-e4d4-d79d79c3230a&t=cc5499f8-bdde-5157-0d76-b6528ecd808e&l=18735

      Attachments

        Activity

          People

            akalashnikov Anton Kalashnikov
            gaoyunhaii Yun Gao
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: