[FLINK-3281] IndexOutOfBoundsException when range-partitioning empty DataSet

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Component/s: Runtime / Coordination, Runtime / Task
• Assignee: Chengxiang Li
• Reporter: Fridtjof Sander
• Created: 24/Jan/16 16:06
• Updated: 02/Feb/16 06:04
• Resolved: 02/Feb/16 06:04

Description

Code:

import org.apache.flink.api.scala._

object RangePartitionOnEmptyDataSet {
    def main(args:Array[String]) = {
      val env = ExecutionEnvironment.getExecutionEnvironment
      env
        .fromCollection(Seq[Tuple1[String]]())
        .partitionByRange(0)
        .collect()
    }
}

Output:

01/24/2016 16:24:36	Job execution switched to status RUNNING.
01/24/2016 16:24:36	DataSource (at RangePartitionOnEmptyDataSet$.main(RangePartitionOnEmptyDataSet.scala:9) (org.apache.flink.api.java.io.CollectionInputFormat))(1/1) switched to SCHEDULED 
01/24/2016 16:24:36	DataSource (at RangePartitionOnEmptyDataSet$.main(RangePartitionOnEmptyDataSet.scala:9) (org.apache.flink.api.java.io.CollectionInputFormat))(1/1) switched to DEPLOYING 
01/24/2016 16:24:36	DataSource (at RangePartitionOnEmptyDataSet$.main(RangePartitionOnEmptyDataSet.scala:9) (org.apache.flink.api.java.io.CollectionInputFormat))(1/1) switched to RUNNING 
01/24/2016 16:24:36	RangePartition: LocalSample(1/1) switched to SCHEDULED 
01/24/2016 16:24:36	RangePartition: LocalSample(1/1) switched to DEPLOYING 
01/24/2016 16:24:36	DataSource (at RangePartitionOnEmptyDataSet$.main(RangePartitionOnEmptyDataSet.scala:9) (org.apache.flink.api.java.io.CollectionInputFormat))(1/1) switched to FINISHED 
01/24/2016 16:24:36	RangePartition: PreparePartition(1/1) switched to SCHEDULED 
01/24/2016 16:24:36	RangePartition: PreparePartition(1/1) switched to DEPLOYING 
01/24/2016 16:24:36	RangePartition: LocalSample(1/1) switched to RUNNING 
01/24/2016 16:24:36	RangePartition: PreparePartition(1/1) switched to RUNNING 
01/24/2016 16:24:36	RangePartition: GlobalSample(1/1) switched to SCHEDULED 
01/24/2016 16:24:36	RangePartition: GlobalSample(1/1) switched to DEPLOYING 
01/24/2016 16:24:36	RangePartition: LocalSample(1/1) switched to FINISHED 
01/24/2016 16:24:36	RangePartition: GlobalSample(1/1) switched to RUNNING 
01/24/2016 16:24:36	RangePartition: Histogram(1/1) switched to SCHEDULED 
01/24/2016 16:24:36	RangePartition: Histogram(1/1) switched to DEPLOYING 
01/24/2016 16:24:36	RangePartition: GlobalSample(1/1) switched to FINISHED 
01/24/2016 16:24:36	RangePartition: Histogram(1/1) switched to RUNNING 
01/24/2016 16:24:37	RangePartition: Histogram(1/1) switched to FAILED 
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
	at java.util.ArrayList.rangeCheck(ArrayList.java:653)
	at java.util.ArrayList.get(ArrayList.java:429)
	at org.apache.flink.runtime.operators.udf.RangeBoundaryBuilder.mapPartition(RangeBoundaryBuilder.java:66)
	at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:98)
	at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:486)
	at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:351)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:561)
	at java.lang.Thread.run(Thread.java:745)

01/24/2016 16:24:37	Job execution switched to status FAILING.
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
	at java.util.ArrayList.rangeCheck(ArrayList.java:653)
	at java.util.ArrayList.get(ArrayList.java:429)
	at org.apache.flink.runtime.operators.udf.RangeBoundaryBuilder.mapPartition(RangeBoundaryBuilder.java:66)
	at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:98)
	at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:486)
	at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:351)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:561)
	at java.lang.Thread.run(Thread.java:745)
01/24/2016 16:24:37	RangePartition: PreparePartition(1/1) switched to CANCELING 
01/24/2016 16:24:37	RangePartition: Partition(1/4) switched to CANCELED 
01/24/2016 16:24:37	RangePartition: Partition(2/4) switched to CANCELED 
01/24/2016 16:24:37	RangePartition: Partition(3/4) switched to CANCELED 
01/24/2016 16:24:37	RangePartition: Partition(4/4) switched to CANCELED 
01/24/2016 16:24:37	CHAIN Partition -> FlatMap (FlatMap at collect(DataSet.scala:542))(1/4) switched to CANCELED 
01/24/2016 16:24:37	CHAIN Partition -> FlatMap (FlatMap at collect(DataSet.scala:542))(2/4) switched to CANCELED 
01/24/2016 16:24:37	CHAIN Partition -> FlatMap (FlatMap at collect(DataSet.scala:542))(3/4) switched to CANCELED 
01/24/2016 16:24:37	CHAIN Partition -> FlatMap (FlatMap at collect(DataSet.scala:542))(4/4) switched to CANCELED 
01/24/2016 16:24:37	RangePartition: PreparePartition(1/1) switched to CANCELED 
01/24/2016 16:24:37	DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@525b461a)(1/4) switched to CANCELED 
01/24/2016 16:24:37	DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@525b461a)(2/4) switched to CANCELED 
01/24/2016 16:24:37	DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@525b461a)(3/4) switched to CANCELED 
01/24/2016 16:24:37	DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@525b461a)(4/4) switched to CANCELED 
01/24/2016 16:24:37	Job execution switched to status FAILED.
Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
	at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$5.apply$mcV$sp(JobManager.scala:570)
	at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$5.apply(JobManager.scala:516)
	at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$5.apply(JobManager.scala:516)
	at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
	at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:41)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:401)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
	at java.util.ArrayList.rangeCheck(ArrayList.java:653)
	at java.util.ArrayList.get(ArrayList.java:429)
	at org.apache.flink.runtime.operators.udf.RangeBoundaryBuilder.mapPartition(RangeBoundaryBuilder.java:66)
	at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:98)
	at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:486)
	at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:351)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:561)
	at java.lang.Thread.run(Thread.java:745)

Process finished with exit code 1

The access happens in RangeBoundaryBuilder.java:66.

Sadly, I don't know enough about this to fix it in reasonable time. chengxiang li maybe?