[SPARK-50077] Introduce a new pattern object for LogicalRelation to help avoiding default pattern (full parameters)

• Type: Task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 4.0.0
• Component/s: SQL
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Jungtaek Lim
• Created: Yesterday 22:28
• Updated: Yesterday 22:33

Description

During my work (work-in-progress), I found it very difficult to add a param into LogicalRelation, because a bunch of places use pattern matching with LogicalRelation, and all of them are using default pattern with specifying all params (despite wildcards).

I looked into the patterns and realized most of the patterns just need relation and catalog table inside the LogicalRelation. output and isStreaming are used infrequently. That said, introducing a new pattern object extracting BaseRelation and CatalogTable from LogicalRelation would cover most of the patterns we have for LogicalRelation, and this would help a lot to introduce a new param into LogicalRelation in future.

Issue Links

• links to: GitHub Pull Request #48605