Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-14087

throws java.lang.ArrayIndexOutOfBoundsException when emiting the data using RebalancePartitioner.

    XMLWordPrintableJSON

Details

    Description

      There is the condition the RecordWriter sharing the ChannelSelector instance.

      When two RecordWriter instance shared the same ChannelSelector Instance , It may throws 

      java.lang.ArrayIndexOutOfBoundsException .  For example,  two recordWriter instance shared the RebalancePartitioner instance. the RebalancePartitioner instance setup 2 number of Channels when the first RecordWriter initializing, next the some RebalancePartitioner instance setup 3 number of  channels When the second RecordWriter initializing. this  throws ArrayIndexOutOfBoundsException when the first RecordWriter instance emits the data.

      The Exception likes

      java.lang.RuntimeException: 2 at org.apache.flink.streaming.runtime.io.RecordWriterOutput.pushToRecordWriter(RecordWriterOutput.java:112) at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:91) at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:47) at org.apache.flink.streaming.runtime.tasks.OperatorChain$BroadcastingOutputCollector.collect(OperatorChain.java:673) at org.apache.flink.streaming.runtime.tasks.OperatorChain$BroadcastingOutputCollector.collect(OperatorChain.java:617) at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:726) at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:699) at org.apache.flink.streaming.api.operators.StreamSourceContexts$NonTimestampContext.collect(StreamSourceContexts.java:104) at com.xx.flink.demo.wordcount.case3.StateTest$Source.run(StateTest.java:107) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:94) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:57) at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:97) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:302) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:734) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ArrayIndexOutOfBoundsException: 2 at org.apache.flink.runtime.io.network.api.writer.RecordWriter.getBufferBuilder(RecordWriter.java:255) at org.apache.flink.runtime.io.network.api.writer.RecordWriter.copyFromSerializerToTargetChannel(RecordWriter.java:177) at org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:162) at org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:128) at org.apache.flink.streaming.runtime.io.RecordWriterOutput.pushToRecordWriter(RecordWriterOutput.java:109) ... 14 more

      Attachments

        Issue Links

          Activity

            People

              gaoyunhaii Yun Gao
              jiangyu jiangyu
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: