[ARROW-4589] [Rust] [DataFusion] Implement projection push down query optimizer rule

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.12.0
• Fix Version/s: 0.13.0
• Component/s: Rust - DataFusion
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/21132
• Assignee: Andy Grove
• Reporter: Andy Grove
• Created: 16/Feb/19 17:42
• Updated: 11/Jan/23 07:34
• Resolved: 17/Feb/19 14:55

Description

If I run a query like the following:

SELECT MIN(fare_amount), MAX(fare_amount) FROM tripdata

I see this logical plan:

Logical plan: Aggregate: groupBy=[[]], aggr=[[MIN(#10), MAX(#10)]]
  TableScan: tripdata projection=None

 

This means that every column is being loaded into arrays rather than just the two columns that I care about, resulting in terrible performance.

Issue Links

• links to: GitHub Pull Request #3664