Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-14316

Stuck in "Job leader ... lost leadership" error

    XMLWordPrintableJSON

    Details

      Description

      This is the first exception caused restart loop. Later exceptions are the same. Job seems to stuck in this permanent failure state.

      2019-10-03 21:42:46,159 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source: clpevents -> device_filter -> processed_imps -> ios_processed_impression -> i
      mps_ts_assigner (449/1360) (d237f5e99b6a4a580498821473763edb) switched from SCHEDULED to FAILED.
      java.lang.Exception: Job leader for job id ecb9ad9be934edf7b1a4f7b9dd6df365 lost leadership.
              at org.apache.flink.runtime.taskexecutor.TaskExecutor$JobLeaderListenerImpl.lambda$jobManagerLostLeadership$1(TaskExecutor.java:1526)
              at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:332)
              at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:158)
              at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.onReceive(AkkaRpcActor.java:142)
              at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)
              at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
              at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
              at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
              at akka.actor.ActorCell.invoke(ActorCell.scala:495)
              at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
              at akka.dispatch.Mailbox.run(Mailbox.scala:224)
              at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
              at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
              at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
              at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
              at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
      

        Attachments

        1. FLINK-14316.tgz
          8.57 MB
          Steven Zhen Wu
        2. RpcConnection.patch
          1 kB
          Piyush Goyal

          Issue Links

            Activity

              People

              • Assignee:
                trohrmann Till Rohrmann
                Reporter:
                stevenz3wu Steven Zhen Wu
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m