Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-14087

throws java.lang.ArrayIndexOutOfBoundsException when emiting the data using RebalancePartitioner.

    XMLWordPrintableJSON

    Details

      Description

      There is the condition the RecordWriter sharing the ChannelSelector instance.

      When two RecordWriter instance shared the same ChannelSelector Instance , It may throws 

      java.lang.ArrayIndexOutOfBoundsException .  For example,  two recordWriter instance shared the RebalancePartitioner instance. the RebalancePartitioner instance setup 2 number of Channels when the first RecordWriter initializing, next the some RebalancePartitioner instance setup 3 number of  channels When the second RecordWriter initializing. this  throws ArrayIndexOutOfBoundsException when the first RecordWriter instance emits the data.

      The Exception likes

      java.lang.RuntimeException: 2 at org.apache.flink.streaming.runtime.io.RecordWriterOutput.pushToRecordWriter(RecordWriterOutput.java:112) at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:91) at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:47) at org.apache.flink.streaming.runtime.tasks.OperatorChain$BroadcastingOutputCollector.collect(OperatorChain.java:673) at org.apache.flink.streaming.runtime.tasks.OperatorChain$BroadcastingOutputCollector.collect(OperatorChain.java:617) at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:726) at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:699) at org.apache.flink.streaming.api.operators.StreamSourceContexts$NonTimestampContext.collect(StreamSourceContexts.java:104) at com.xx.flink.demo.wordcount.case3.StateTest$Source.run(StateTest.java:107) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:94) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:57) at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:97) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:302) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:734) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ArrayIndexOutOfBoundsException: 2 at org.apache.flink.runtime.io.network.api.writer.RecordWriter.getBufferBuilder(RecordWriter.java:255) at org.apache.flink.runtime.io.network.api.writer.RecordWriter.copyFromSerializerToTargetChannel(RecordWriter.java:177) at org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:162) at org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:128) at org.apache.flink.streaming.runtime.io.RecordWriterOutput.pushToRecordWriter(RecordWriterOutput.java:109) ... 14 more

        Attachments

        1. image-2019-09-16-19-14-39-403.png
          27 kB
          luojiangyu
        2. image-2019-09-16-19-15-34-639.png
          27 kB
          luojiangyu

          Issue Links

            Activity

              People

              • Assignee:
                gaoyunhaii Yun Gao
                Reporter:
                jiangyu luojiangyu
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: