[FLINK-14087] throws java.lang.ArrayIndexOutOfBoundsException when emiting the data using RebalancePartitioner.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.8.0, 1.8.1, 1.9.0
• Fix Version/s: 1.10.3, 1.11.2, 1.12.0
• Component/s: API / DataStream, Runtime / Network
• Labels: pull-request-available
• Assignee: Yun Gao
• Reporter: jiangyu
• Created: 16/Sep/19 09:38
• Updated: 04/Sep/20 06:58
• Resolved: 04/Sep/20 03:04

Description

There is the condition the RecordWriter sharing the ChannelSelector instance.

When two RecordWriter instance shared the same ChannelSelector Instance , It may throws 

java.lang.ArrayIndexOutOfBoundsException .  For example,  two recordWriter instance shared the RebalancePartitioner instance. the RebalancePartitioner instance setup 2 number of Channels when the first RecordWriter initializing, next the some RebalancePartitioner instance setup 3 number of  channels When the second RecordWriter initializing. this  throws ArrayIndexOutOfBoundsException when the first RecordWriter instance emits the data.

The Exception likes

java.lang.RuntimeException: 2 at org.apache.flink.streaming.runtime.io.RecordWriterOutput.pushToRecordWriter(RecordWriterOutput.java:112) at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:91) at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:47) at org.apache.flink.streaming.runtime.tasks.OperatorChain$BroadcastingOutputCollector.collect(OperatorChain.java:673) at org.apache.flink.streaming.runtime.tasks.OperatorChain$BroadcastingOutputCollector.collect(OperatorChain.java:617) at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:726) at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:699) at org.apache.flink.streaming.api.operators.StreamSourceContexts$NonTimestampContext.collect(StreamSourceContexts.java:104) at com.xx.flink.demo.wordcount.case3.StateTest$Source.run(StateTest.java:107) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:94) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:57) at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:97) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:302) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:734) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ArrayIndexOutOfBoundsException: 2 at org.apache.flink.runtime.io.network.api.writer.RecordWriter.getBufferBuilder(RecordWriter.java:255) at org.apache.flink.runtime.io.network.api.writer.RecordWriter.copyFromSerializerToTargetChannel(RecordWriter.java:177) at org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:162) at org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:128) at org.apache.flink.streaming.runtime.io.RecordWriterOutput.pushToRecordWriter(RecordWriterOutput.java:109) ... 14 more

Issue Links

• links to: GitHub Pull Request #13249