[ARROW-9771] [Rust] [DataFusion] Predicate Pushdown Improvement: treat predicates separated by AND separately

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Component/s: Rust, Rust - DataFusion
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/25820
• Assignee: Jorge Leitão
• Reporter: Andrew Lamb
• Created: 17/Aug/20 14:42
• Updated: 11/Jan/23 08:08
• Resolved: 14/Dec/20 18:47

Description

As discussed by jorgecarleitao and houqp here: https://github.com/apache/arrow/pull/7880#pullrequestreview-468057624

If a predicate is a conjunction (aka AND'd) together, each of the clauses can be treated separately (e.g. a single filter expression A > 5 And B < 4 can be broken up and each of A > 5 and B < 4 can be potentially pushed down different levels

The filter pushdown logic works for the following case (when a and b are are separate selections, predicate for a is pushed below the Aggregate in the optimized plan):

********Original plan:
Selection: #b GtEq Int64(1)
  Selection: #a LtEq Int64(1)
    Aggregate: groupBy=[[#a]], aggr=[[MIN(#b)]]
      TableScan: test projection=None

********Optimized plan:
Selection: #b GtEq Int64(1)
  Aggregate: groupBy=[[#a]], aggr=[[MIN(#b)]]
    Selection: #a LtEq Int64(1)
      TableScan: test projection=None

But not for this when a and b are AND'd together

********Original plan:
Selection: #a LtEq Int64(1) And #b GtEq Int64(1)
  Aggregate: groupBy=[[#a]], aggr=[[MIN(#b)]]
    TableScan: test projection=None
********Optimized plan:
Selection: #a LtEq Int64(1) And #b GtEq Int64(1)
  Aggregate: groupBy=[[#a]], aggr=[[MIN(#b)]]
    TableScan: test projection=None

Issue Links

• links to: GitHub Pull Request #8903