[SPARK-26063] CatalystDataToAvro gives "UnresolvedException: Invalid call to dataType on unresolved object" when requested for numberedTreeString

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.4.0
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Jacek Laskowski
• Created: 14/Nov/18 11:13
• Updated: 16/Nov/18 03:11
• Resolved: 16/Nov/18 03:11

Description

The following gives org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid call to dataType on unresolved object, tree: 'id:

// ./bin/spark-shell --packages org.apache.spark:spark-avro_2.12:2.4.0
scala> spark.version
res0: String = 2.4.0

import org.apache.spark.sql.avro._
val q = spark.range(1).withColumn("to_avro_id", to_avro('id))
val logicalPlan = q.queryExecution.logical

scala> logicalPlan.expressions.drop(1).head.numberedTreeString
org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid call to dataType on unresolved object, tree: 'id
at org.apache.spark.sql.catalyst.analysis.UnresolvedAttribute.dataType(unresolved.scala:105)
at org.apache.spark.sql.avro.CatalystDataToAvro.simpleString(CatalystDataToAvro.scala:56)
at org.apache.spark.sql.catalyst.expressions.Expression.verboseString(Expression.scala:233)
at org.apache.spark.sql.catalyst.trees.TreeNode.generateTreeString(TreeNode.scala:548)
at org.apache.spark.sql.catalyst.trees.TreeNode.generateTreeString(TreeNode.scala:569)
at org.apache.spark.sql.catalyst.trees.TreeNode.treeString(TreeNode.scala:472)
at org.apache.spark.sql.catalyst.trees.TreeNode.treeString(TreeNode.scala:469)
at org.apache.spark.sql.catalyst.trees.TreeNode.numberedTreeString(TreeNode.scala:483)
... 51 elided

Issue Links

• duplicates: SPARK-25883 Override method `prettyName` in `from_avro`/`to_avro`