[FLINK-32945] NullPointerException when executing TopSpeedWindowing example with checkpointing enabled

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.18.0
• Fix Version/s: 1.18.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: Jiang Xin
• Reporter: Matthias Pohl
• Created: 23/Aug/23 09:41
• Updated: 30/Aug/23 10:09
• Resolved: 30/Aug/23 10:09

Description

I tried running the TopSpeedWindowing example with a checkpoint interval of 10s and the data input file that I attached to this Jira. I run into the following NullPointerException:

2023-08-23 09:56:03,111 ERROR org.apache.flink.runtime.rpc.pekko.FencedPekkoRpcActor       [] - Error while executing remote procedure call public void org.apache.flink.runtime.jobmaster.JobMaster.notifyEndOfData(org.apache.flink.runtime.executiongraph.ExecutionAttemptID).
java.lang.reflect.InvocationTargetException: null
	at jdk.internal.reflect.GeneratedMethodAccessor20.invoke(Unknown Source) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
	at org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.lambda$handleRpcInvocation$0(PekkoRpcActor.java:301) ~[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at org.apache.flink.runtime.concurrent.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83) ~[classes/:?]
	at org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleRpcInvocation(PekkoRpcActor.java:300) ~[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleRpcMessage(PekkoRpcActor.java:222) ~[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at org.apache.flink.runtime.rpc.pekko.FencedPekkoRpcActor.handleRpcMessage(FencedPekkoRpcActor.java:85) ~[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleMessage(PekkoRpcActor.java:168) ~[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) [flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) [flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at scala.PartialFunction.applyOrElse(PartialFunction.scala:127) [flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126) [flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) [flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:175) [flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176) [flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176) [flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) [flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) [flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229) [flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) [flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) [flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) [flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) [flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) [flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) [?:?]
	at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) [?:?]
	at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) [?:?]
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) [?:?]
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) [?:?]
Caused by: java.lang.NullPointerException: Cannot invoke "java.util.BitSet.set(int)" because "subtaskStatus" is null
	at org.apache.flink.runtime.scheduler.VertexEndOfDataListener.recordTaskEndOfData(VertexEndOfDataListener.java:51) ~[classes/:?]
	at org.apache.flink.runtime.scheduler.SchedulerBase.notifyEndOfData(SchedulerBase.java:1079) ~[classes/:?]
	at org.apache.flink.runtime.jobmaster.JobMaster.notifyEndOfData(JobMaster.java:508) ~[classes/:?]
	... 30 more

I adapted the TopSpeedWindowing example slightly (see gist) and executed it from within the IDE with the attached input data.

Issue Links

• Discovered while testing: FLINK-32911 Release Testing: Verify FLINK-28386: Trigger an immediate checkpoint after all sinks finished
• is caused by: FLINK-28386 Trigger an immediate checkpoint after all tasks has reached end-of-data
• links to: GitHub Pull Request #23268
• links to: GitHub Pull Request #23315
• mentioned in: Page Loading...