Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-16819

Got KryoException while using UDAF in flink1.9

    XMLWordPrintableJSON

    Details

      Description

      Recently,  we are trying to upgrade online sql jobs from flink1.7 to flink1.9 , most jobs works fine, but some jobs got  KryoExceptions. 

      We found that UDAF will trigger this exception, btw ,we are using blink planner.

      Here is the full stack traces:
      2020-03-27 11:46:55
      com.esotericsoftware.kryo.KryoException: java.lang.IndexOutOfBoundsException: Index: 104, Size: 2
      Serialization trace:
      seed (java.util.Random)
      gen (com.tdunning.math.stats.AVLTreeDigest)
          at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
          at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
          at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
          at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
          at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
          at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761)
          at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:346)
          at org.apache.flink.util.InstantiationUtil.deserializeFromByteArray(InstantiationUtil.java:536)
          at org.apache.flink.table.dataformat.BinaryGeneric.getJavaObjectFromBinaryGeneric(BinaryGeneric.java:86)
          at org.apache.flink.table.dataformat.DataFormatConverters$GenericConverter.toExternalImpl(DataFormatConverters.java:628)
          at org.apache.flink.table.dataformat.DataFormatConverters$GenericConverter.toExternalImpl(DataFormatConverters.java:633)
          at org.apache.flink.table.dataformat.DataFormatConverters$DataFormatConverter.toExternal(DataFormatConverters.java:320)
          at org.apache.flink.table.dataformat.DataFormatConverters$PojoConverter.toExternalImpl(DataFormatConverters.java:1293)
          at org.apache.flink.table.dataformat.DataFormatConverters$PojoConverter.toExternalImpl(DataFormatConverters.java:1257)
          at org.apache.flink.table.dataformat.DataFormatConverters$DataFormatConverter.toExternal(DataFormatConverters.java:302)
          at GroupAggsHandler$71.setAccumulators(Unknown Source)
          at org.apache.flink.table.runtime.operators.aggregate.GroupAggFunction.processElement(GroupAggFunction.java:151)
          at org.apache.flink.table.runtime.operators.aggregate.GroupAggFunction.processElement(GroupAggFunction.java:43)
          at org.apache.flink.streaming.api.operators.KeyedProcessOperator.processElement(KeyedProcessOperator.java:85)
          at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processElement(StreamOneInputProcessor.java:164)
          at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:143)
          at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:279)
          at org.apache.flink.streaming.runtime.tasks.StreamTask.run(StreamTask.java:301)
          at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:406)
          at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705)
          at org.apache.flink.runtime.taskmanager.Task.run(Task.java:530)
          at java.lang.Thread.run(Thread.java:748)
      Caused by: java.lang.IndexOutOfBoundsException: Index: 104, Size: 2
          at java.util.ArrayList.rangeCheck(ArrayList.java:657)
          at java.util.ArrayList.get(ArrayList.java:433)
          at com.esotericsoftware.kryo.util.MapReferenceResolver.getReadObject(MapReferenceResolver.java:42)
          at com.esotericsoftware.kryo.Kryo.readReferenceOrNull(Kryo.java:805)
          at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:677)
          at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
          ... 26 more

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              dixingxing@yeah.net Xingxing Di
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: