Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-8813

AutoParallellismITCase fails with Flip6

    XMLWordPrintableJSON

    Details

      Description

      The AutoParallelismITCase fails when running against flip6. (https://travis-ci.org/zentol/flink/jobs/347373854)

      It appears that the JobMaster does not properly handle ExecutionConfig#PARALLELISM_AUTO_MAX.

       

      Exception:

      
      Caused by: org.apache.flink.runtime.client.JobSubmissionException: Could not start JobManager.
      	at org.apache.flink.runtime.dispatcher.Dispatcher.submitJob(Dispatcher.java:287)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:210)
      	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:154)
      	at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleMessage(FencedAkkaRpcActor.java:66)
      	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$onReceive$1(AkkaRpcActor.java:132)
      	at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)
      	at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
      	at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
      	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
      	at akka.actor.ActorCell.invoke(ActorCell.scala:495)
      	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
      	at akka.dispatch.Mailbox.run(Mailbox.scala:224)
      	at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
      	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
      	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
      	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
      	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
      Caused by: org.apache.flink.runtime.client.JobExecutionException: Could not set up JobManager
      	at org.apache.flink.runtime.jobmaster.JobManagerRunner.<init>(JobManagerRunner.java:181)
      	at org.apache.flink.runtime.dispatcher.Dispatcher$DefaultJobManagerRunnerFactory.createJobManagerRunner(Dispatcher.java:747)
      	at org.apache.flink.runtime.dispatcher.Dispatcher.submitJob(Dispatcher.java:243)
      	... 20 more
      Caused by: java.lang.IllegalArgumentException: The parallelism must be at least one.
      	at org.apache.flink.runtime.jobgraph.JobVertex.setParallelism(JobVertex.java:290)
      	at org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:162)
      	at org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:295)
      	at org.apache.flink.runtime.jobmaster.JobManagerRunner.<init>(JobManagerRunner.java:170)
      	... 22 more

       

      The likely culprit is this call to ExecutionGraphBuilder#buildGraph in the JobMaster constructor:

      this.executionGraph = ExecutionGraphBuilder.buildGraph(
         null,
         jobGraph,
         jobMasterConfiguration.getConfiguration(),
         scheduledExecutorService,
         scheduledExecutorService,
         slotPool.getSlotProvider(),
         userCodeLoader,
         highAvailabilityServices.getCheckpointRecoveryFactory(),
         rpcTimeout,
         restartStrategy,
         jobMetricGroup,
         -1, // parallelismForAutoMax
         blobServer,
         jobMasterConfiguration.getSlotRequestTimeout(),
         log);

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                pnowojski Piotr Nowojski
                Reporter:
                chesnay Chesnay Schepler
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: