[FLINK-2338] Shut down "Storm Topologies" cleanly

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Component/s: Legacy Components / Storm Compatibility
• Assignee: Matthias J. Sax
• Reporter: Matthias J. Sax
• Created: 09/Jul/15 12:23
• Updated: 31/Mar/16 22:40
• Resolved: 31/Mar/16 22:40

Description

Currently, it is not possible to stop a Flink streaming program in a clean way. Thus, emulating Storm's "kill" command is done the "hard way" resulting in the following exception shown in the log:

org.apache.flink.runtime.client.JobExecutionException: Communication with JobManager failed: Lost connection to JobManager akka://flink/user/jobmanager
at org.apache.flink.runtime.client.JobClient.submitJobAndWait(JobClient.java:169)
at org.apache.flink.runtime.minicluster.FlinkMiniCluster.submitJobAndWait(FlinkMiniCluster.scala:205)
at org.apache.flink.runtime.minicluster.FlinkMiniCluster.submitJobAndWait(FlinkMiniCluster.scala:195)
at org.apache.flink.streaming.util.TestStreamEnvironment$1.run(TestStreamEnvironment.java:116)
Caused by: java.lang.Exception: Lost connection to JobManager akka://flink/user/jobmanager
at org.apache.flink.runtime.client.JobClientActor.onReceive(JobClientActor.java:131)
at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167)
at akka.actor.Actor$class.aroundReceive(Actor.scala:465)
at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:97)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
at akka.actor.dungeon.DeathWatch$class.receivedTerminated(DeathWatch.scala:46)
at akka.actor.ActorCell.receivedTerminated(ActorCell.scala:369)
at akka.actor.ActorCell.autoReceiveMessage(ActorCell.scala:501)
at akka.actor.ActorCell.invoke(ActorCell.scala:486)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254)
at akka.dispatch.Mailbox.run(Mailbox.scala:221)
at akka.dispatch.Mailbox.exec(Mailbox.scala:231)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

The exception is expected currently. However, a clean "kill" is preferable. This can done after the new STOP signal is available (https://issues.apache.org/jira/browse/FLINK-2111).

Issue Links

• Is contained by: FLINK-2111 Add "stop" signal to cleanly shutdown streaming jobs