Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-22053

NumberSequenceSource causes fatal exception when less splits than parallelism.

    XMLWordPrintableJSON

    Details

      Description

      If more splits than

      Caused by: java.lang.IllegalArgumentException: 'from' must be <= 'to'
      	at org.apache.flink.util.Preconditions.checkArgument(Preconditions.java:138) ~[flink-core-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
      	at org.apache.flink.api.connector.source.lib.NumberSequenceSource$NumberSequenceSplit.<init>(NumberSequenceSource.java:148) ~[flink-core-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
      	at org.apache.flink.api.connector.source.lib.NumberSequenceSource.createEnumerator(NumberSequenceSource.java:111) ~[flink-core-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
      	at org.apache.flink.runtime.source.coordinator.SourceCoordinator.start(SourceCoordinator.java:126) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
      	at org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator$DeferrableCoordinator.applyCall(RecreateOnResetOperatorCoordinator.java:296) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
      	at org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator.start(RecreateOnResetOperatorCoordinator.java:71) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
      	at org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder.start(OperatorCoordinatorHolder.java:182) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
      	at org.apache.flink.runtime.scheduler.DefaultOperatorCoordinatorHandler.startAllOperatorCoordinators(DefaultOperatorCoordinatorHandler.java:85) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
      	at org.apache.flink.runtime.scheduler.SchedulerBase.startScheduling(SchedulerBase.java:501) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
      	at org.apache.flink.runtime.jobmaster.JobMaster.startScheduling(JobMaster.java:955) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
      	at org.apache.flink.runtime.jobmaster.JobMaster.startJobExecution(JobMaster.java:873) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
      	at org.apache.flink.runtime.jobmaster.JobMaster.onStart(JobMaster.java:383) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
      	at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStart(RpcEndpoint.java:181) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
      	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StoppedState.start(AkkaRpcActor.java:605) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
      	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:180) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
      	at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[akka-actor_2.11-2.5.21.jar:2.5.21]
      	at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[akka-actor_2.11-2.5.21.jar:2.5.21]
      	at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[scala-library-2.11.12.jar:?]
      	at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[akka-actor_2.11-2.5.21.jar:2.5.21]
      	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) ~[scala-library-2.11.12.jar:?]
      	... 12 more
      

      To reproduce

          @Test
          public void testLessSplitsThanParallelism() throws Exception {
              StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
              env.setParallelism(12);
              env.fromSequence(0, 10);
              env.execute();
          }
      

        Attachments

          Activity

            People

            • Assignee:
              AHeise Arvid Heise
              Reporter:
              AHeise Arvid Heise
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: