Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-44702

Cannot derive ExpressionEncoder when using types tagged by a trait

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 3.1.3, 3.2.4, 3.3.2
    • None
    • SQL

    Description

      Steps to reproduce:

      trait Tag
      case class Foo(x: Int)
      case class Bar(x: Foo with Tag)
      ExpressionEncoder.apply[Bar]()

      the ExpressionEncoder throws the following error (this was for 3.1.2):

      scala.MatchError: Foo with Tag (of class scala.reflect.internal.Types$RefinedType0)
        at org.apache.spark.sql.catalyst.ScalaReflection.getConstructorParameters(ScalaReflection.scala:931)
        at org.apache.spark.sql.catalyst.ScalaReflection.getConstructorParameters$(ScalaReflection.scala:928)
        at org.apache.spark.sql.catalyst.ScalaReflection$.getConstructorParameters(ScalaReflection.scala:49) 

      The bug is on this line (on master).

      val TypeRef(_, _, actualTypeArgs) = dealiasedTpe 

      which is an incomplete pattern match. The pattern match needs be extended to include matching on RefinedType as well.

      Attachments

        Activity

          People

            Unassigned Unassigned
            ben_hemsi Ben Hemsi
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: