[SPARK-46640] RemoveRedundantAliases does not account for SubqueryExpression when removing aliases

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0, 3.5.1
• Component/s: Optimizer
• Labels: pull-request-available
• Assignee: Nikhil Sheoran
• Reporter: Nikhil Sheoran
• Created: 09/Jan/24 22:45
• Updated: 12/Jan/24 02:27
• Resolved: 12/Jan/24 02:26

Description

`RemoveRedundantAliases` does not take into account the outer attributes of a `SubqueryExpression` aliases, potentially removing them if it thinks they are redundant.

This can cause scenarios where a subquery expression has conditions like `a#x = a#x` i.e. both the attribute names and the expression ID(s) are the same. This can then lead to conflicting expression ID(s) error.

In `RemoveRedundantAliases`, we have an excluded AttributeSet argument denoting the references for which we should not remove aliases. For a query with a subquery expression, adding the references of this subquery in the excluded set prevents such rewrite from happening.

Issue Links

• links to: GitHub Pull Request #44645