[SPARK-19544] Improve error message when some column types are compatible and others are not in set/union operations

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.2.0
• Fix Version/s: 2.2.0
• Component/s: SQL
• Assignee: Hyukjin Kwon
• Reporter: Hyukjin Kwon
• Created: 10/Feb/17 04:50
• Updated: 12/Dec/22 17:50
• Resolved: 13/Feb/17 15:08

Description

Currently,

Seq((1,("a", 1))).toDF union Seq((1L,("a", "b"))).toDF
org.apache.spark.sql.AnalysisException: Union can only be performed on tables with the compatible column types. LongType <> IntegerType at the first column of the second table;;

It prints that it fails due to LongType <> IntegerType whereas actually it is StructType(...) <> StructType(...)

Issue Links

• links to: [Github] Pull Request #16882 (HyukjinKwon)