[SPARK-47398] AQE doesn't allow for extension of InMemoryTableScanExec

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.5.0, 3.5.1
• Fix Version/s: 4.0.0, 3.5.2
• Component/s: SQL
• Labels: pull-request-available
• Assignee: Raza Jafri
• Reporter: Raza Jafri
• Shepherd: Robert Joseph Evans
• Created: 14/Mar/24 15:21
• Updated: 22/Mar/24 04:25
• Resolved: 21/Mar/24 19:49

Description

As part of SPARK-42101, we added support to AQE for handling InMemoryTableScanExec. 

This change directly references `InMemoryTableScanExec` which limits users from extending the caching functionality that was added as part of SPARK-32274 

Issue Links

• links to: GitHub Pull Request #45525