[SPARK-38918] Nested column pruning should filter out attributes that do not belong to the current relation

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.1.3, 3.0.4, 3.3.0, 3.4.0
• Component/s: SQL
• Assignee: Allison Wang
• Reporter: Allison Wang
• Created: 15/Apr/22 20:47
• Updated: 06/Dec/22 21:46
• Resolved: 27/Apr/22 06:50

Description

`SchemaPruning` currently does not check if the root field of a nested column belongs to the current relation. This can happen when the filter contains correlated subqueries, where the children field can contain attributes from both the inner and the outer query.

Issue Links

• links to: [Github] Pull Request #36216 (allisonwang-db)
• links to: [Github] Pull Request #36216 (allisonwang-db)
• links to: [Github] Pull Request #36386 (allisonwang-db)
• links to: [Github] Pull Request #36386 (allisonwang-db)
• links to: [Github] Pull Request #36387 (allisonwang-db)
• links to: [Github] Pull Request #36387 (allisonwang-db)
• links to: [Github] Pull Request #36388 (allisonwang-db)
• links to: [Github] Pull Request #36388 (allisonwang-db)