[SPARK-18717] Datasets - crash (compile exception) when mapping to immutable scala map

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.2, 2.1.0
• Fix Version/s: 2.1.1, 2.2.0
• Assignee: Andrew Ray
• Reporter: Damian Momot
• Created: 05/Dec/16 13:09
• Updated: 11/Feb/17 00:11
• Resolved: 13/Dec/16 07:50

Description

val spark: SparkSession = ???

case class Test(id: String, map_test: Map[Long, String])

spark.sql("CREATE TABLE xyz.map_test (id string, map_test map<int, string>) STORED AS PARQUET")

spark.sql("SELECT * FROM xyz.map_test").as[Test].map(t => t).collect()

org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 307, Column 108: No applicable constructor/method found for actual parameters "java.lang.String, scala.collection.Map"; candidates are: "$line14.$read$$iw$$iw$Test(java.lang.String, scala.collection.immutable.Map)"

Issue Links

• is duplicated by: SPARK-14767 Codegen "no constructor found" errors with Maps inside case classes in Datasets
• links to: [Github] Pull Request #16161 (aray)