Major Description
A TreeNodeException is thrown when executing the following minimal example in Spark 2.0. Crucial is that the column q is generated with lit/expr.
import spark.implicits._ case class test (x: Int, q: Int) val d = Seq(1).toDF("x") d.withColumn("q", lit(0)).as[test].groupByKey(_.x).flatMapGroups{case (x, iter) => List()}.show d.withColumn("q", expr("0")).as[test].groupByKey(_.x).flatMapGroups{case (x, iter) => List()}.show // this works fine d.withColumn("q", lit(0)).as[test].groupByKey(_.x).count()
The exception is: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: Binding attribute, tree: x#5
A possible workaround is to write the dataframe to disk before grouping and mapping.
