[SPARK-32372] "Resolved attribute(s) XXX missing" after dudup conflict references

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.2, 2.3.4, 2.4.6, 3.0.0
• Fix Version/s: 2.4.7, 3.0.1, 3.1.0
• Component/s: SQL
• Assignee: wuyi
• Reporter: wuyi
• Created: 20/Jul/20 14:48
• Updated: 24/Jul/20 04:27
• Resolved: 23/Jul/20 14:38

Description

// case class Person(id: Int, name: String, age: Int)

sql("SELECT name, avg(age) as avg_age FROM person GROUP BY name").createOrReplaceTempView("person_a")
sql("SELECT p1.name, p2.avg_age FROM person p1 JOIN person_a p2 ON p1.name = p2.name").createOrReplaceTempView("person_b")
sql("SELECT * FROM person_a UNION SELECT * FROM person_b")   .createOrReplaceTempView("person_c")
sql("SELECT p1.name, p2.avg_age FROM person_c p1 JOIN person_c p2 ON p1.name = p2.name").show

error:

[info]   Failed to analyze query: org.apache.spark.sql.AnalysisException: Resolved attribute(s) avg_age#235 missing from name#233,avg_age#231 in operator !Project [name#233, avg_age#235]. Attribute(s) with the same name appear in the operation: avg_age. Please check if the right attribute(s) are used.;;
...

Issue Links

• links to: [Github] Pull Request #29166 (Ngone51)
• links to: [Github] Pull Request #29166 (Ngone51)
• links to: [Github] Pull Request #29208 (Ngone51)