[SPARK-32943] Add improvement to filter pushdown and project combination

• Type: Umbrella
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.1.0
• Component/s: SQL
• Assignee: L. C. Hsieh
• Reporter: L. C. Hsieh
• Created: 20/Sep/20 01:51
• Updated: 08/Dec/20 01:11
• Resolved: 08/Dec/20 01:11

Description

We observed in our customer payloads that existing optimization of filter pushdown and project combination produces query plan with bad performance in some cases. This is an umbrella for some tasks to improve filter pushdown and project combination.

Sub-Tasks

• Avoid push down Filter through Project when it will hurts performance: Resolved
• Avoid project combination when it will hurst performance: Resolved