[SPARK-14767] Codegen "no constructor found" errors with Maps inside case classes in Datasets

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Duplicate
• Affects Version/s: 2.0.0
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Burak Yavuz
• Created: 20/Apr/16 22:24
• Updated: 14/Dec/16 23:41
• Resolved: 14/Dec/16 23:41

Description

When I have a `Map` inside a case class and am trying to use Datasets,
the simplest operation throws an exception, because the generated code is looking for a constructor with `scala.collection.Map` whereas the constructor takes `scala.collection.immutable.Map`.

To reproduce:

case class Bug(bug: Map[String, String])

val ds = Seq(Bug(Map("name" -> "dummy"))).toDS()

ds.map { b =>
  b.bug.getOrElse("name", null)
}.count()

Stacktrace:

Caused by: java.util.concurrent.ExecutionException: java.lang.Exception: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 163, Column 150: No applicable constructor/method found for actual parameters "scala.collection.Map"; candidates are: Bug(scala.collection.immutable.Map)"

Issue Links

• duplicates: SPARK-18717 Datasets - crash (compile exception) when mapping to immutable scala map