Details
-
Improvement
-
Status: Open
-
Not a Priority
-
Resolution: Unresolved
-
1.11.0
-
None
Description
Currently, when flink serialization system spots an ADT-style class hierarchy in the Scala code, it falls back to GenericType and kryo serialization, which may introduce performance issues. For example, for code:
sealed trait ADT
case class Foo(a: String) extends ADT
case class Bar(b: Int) extends ADT
env.fromCollection(List[ADT](Foo("a"),Bar(1))).collect()
It will fall back to Kryo even if there is no problem with dealing with List[Foo] or List[Bar] separately. Using ADTs is a convenient way in Scala to model different types of messages, but Flink type system performance limits it to only a non performance-critical paths.
It would be nice to have a sealed trait hierarchies support out of the box without kryo fallback.