Description
Encoders treat all collections the same (i.e. Seq vs List) which force users to only define classes with the most generic type.
An example error:
case class SpecificCollection(aList: List[Int]) Seq(SpecificCollection(1 :: Nil)).toDS().collect()
java.lang.RuntimeException: Error while decoding: java.util.concurrent.ExecutionException: java.lang.Exception: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 98, Column 120: No applicable constructor/method found for actual parameters "scala.collection.Seq"; candidates are: "line29e7e4b1e36445baa3505b2e102aa86b29.$read$$iw$$iw$$iw$$iw$SpecificCollection(scala.collection.immutable.List)"
Attachments
Issue Links
- is duplicated by
-
SPARK-19104 CompileException with Map and Case Class in Spark 2.1.0
- Resolved
-
SPARK-19434 Dataframe/Dataset unserialization failing with Map
- Resolved
- relates to
-
SPARK-19089 Support nested arrays/seqs in Datasets
- Resolved
- links to