[FLINK-1391] Kryo fails to properly serialize avro collection types

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.8.0, 0.9
• Fix Version/s: 0.9, 0.8.1
• Assignee: Robert Metzger
• Reporter: Robert Metzger
• Created: 12/Jan/15 17:38
• Updated: 03/Aug/16 01:24
• Resolved: 18/Feb/15 15:40

Description

Before FLINK-610, Avro was the default generic serializer.
Now, special types coming from Avro are handled by Kryo .. which seems to cause errors like:

Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: java.lang.NullPointerException
	at org.apache.avro.generic.GenericData$Array.add(GenericData.java:200)
	at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:116)
	at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:22)
	at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761)
	at org.apache.flink.api.java.typeutils.runtime.KryoSerializer.deserialize(KryoSerializer.java:143)
	at org.apache.flink.api.java.typeutils.runtime.KryoSerializer.deserialize(KryoSerializer.java:148)
	at org.apache.flink.api.java.typeutils.runtime.PojoSerializer.deserialize(PojoSerializer.java:244)
	at org.apache.flink.runtime.plugable.DeserializationDelegate.read(DeserializationDelegate.java:56)
	at org.apache.flink.runtime.io.network.serialization.AdaptiveSpanningRecordDeserializer.getNextRecord(AdaptiveSpanningRecordDeserializer.java:71)
	at org.apache.flink.runtime.io.network.channels.InputChannel.readRecord(InputChannel.java:189)
	at org.apache.flink.runtime.io.network.gates.InputGate.readRecord(InputGate.java:176)
	at org.apache.flink.runtime.io.network.api.MutableRecordReader.next(MutableRecordReader.java:51)
	at org.apache.flink.runtime.operators.util.ReaderIterator.next(ReaderIterator.java:53)
	at org.apache.flink.runtime.operators.DataSinkTask.invoke(DataSinkTask.java:170)
	at org.apache.flink.runtime.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:257)
	at java.lang.Thread.run(Thread.java:744)

Issue Links

• is related to: FLINK-610 Replace Avro serialization by Kryo
• is related to: SPARK-3601 Kryo NPE for output operations on Avro complex Objects even after registering.