[SPARK-45892] Refactor the optimizer plan validation to decouple validateSchemaOutput and validateExprIdUniqueness

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0, 3.4.1
• Fix Version/s: 4.0.0
• Component/s: Optimizer
• Labels: pull-request-available
• Assignee: Xi Liang
• Reporter: Xi Liang
• Created: 10/Nov/23 22:26
• Updated: 13/Nov/23 21:28
• Resolved: 13/Nov/23 21:28

Description

Currently, the expressionIDUniqueness validation is closely coupled with output schema validation. 

https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala#L403C7-L411C8

Some refactoring can improve readability and reuse.

Issue Links

• links to: GitHub Pull Request #43761