[SPARK-39376] Do not output duplicated columns in star expansion of subquery alias of NATURAL/USING JOIN

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.3.0, 3.2.2
• Component/s: SQL
• Assignee: Karen Feng
• Reporter: Karen Feng
• Created: 03/Jun/22 18:16
• Updated: 30/Aug/22 15:49
• Resolved: 06/Jun/22 13:00

Description

A bug was introduced in https://issues.apache.org/jira/browse/SPARK-34527 such that the duplicated columns within a NATURAL/USING JOIN were output from the qualified star of a subquery alias. For example:

val df1 = Seq((3, 8)).toDF("a", "b") 
val df2 = Seq((8, 7)).toDF("b", "d") 
val joinDF = df1.join(df2, "b")
joinDF.alias("r").select("r.*")

Outputs two duplicate `b` columns, instead of just one.

Issue Links

• is duplicated by: SPARK-38603 Qualified star selection produces duplicated common columns after join then alias
• links to: [Github] Pull Request #36763 (karenfeng)