[SPARK-33621] Add a way to inject data source rewrite rules

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 3.1.0
• Component/s: SQL
• Assignee: Anton Okolnychyi
• Reporter: Anton Okolnychyi
• Created: 01/Dec/20 11:16
• Updated: 24/Dec/20 20:37
• Resolved: 07/Dec/20 23:32

Description

SparkSessionExtensions allow us to inject optimization rules but they are added to operator optimization batch. There are cases when users need to run rules after the operator optimization batch. Currently, this is not possible.

Issue Links

• is related to: SPARK-33612 Add dataSourceRewriteRules batch to Optimizer
• links to: [Github] Pull Request #30577 (aokolnychyi)
• links to: [Github] Pull Request #30577 (aokolnychyi)
• links to: [Github] Pull Request #30917 (aokolnychyi)
• links to: [Github] Pull Request #30917 (aokolnychyi)