[SPARK-38347] Nullability propagation in transformUpWithNewOutput

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.3.0, 3.2.2
• Component/s: SQL
• Assignee: Yingyi Bu
• Reporter: Yingyi Bu
• Created: 28/Feb/22 09:14
• Updated: 01/Mar/22 01:40
• Resolved: 01/Mar/22 01:40

Description

The nullability of a replaced attribute should be `a.nullable` instead of `b.nullable`:
https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala#L357

The scenario is a left outer join where the RHS attributes are replaced bottom up.

Issue Links

• links to: [Github] Pull Request #35677 (sigmod)
• links to: [Github] Pull Request #35685 (sigmod)