Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-18717

Datasets - crash (compile exception) when mapping to immutable scala map

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.2, 2.1.0
    • Fix Version/s: 2.1.1, 2.2.0
    • Component/s: None
    • Labels:
      None

      Description

      val spark: SparkSession = ???
      
      case class Test(id: String, map_test: Map[Long, String])
      
      spark.sql("CREATE TABLE xyz.map_test (id string, map_test map<int, string>) STORED AS PARQUET")
      
      spark.sql("SELECT * FROM xyz.map_test").as[Test].map(t => t).collect()
      
      org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 307, Column 108: No applicable constructor/method found for actual parameters "java.lang.String, scala.collection.Map"; candidates are: "$line14.$read$$iw$$iw$Test(java.lang.String, scala.collection.immutable.Map)"
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                a1ray Andrew Ray
                Reporter:
                Daimon Damian Momot
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: