[BEAM-8468] Add predicate/filter push-down capability to IO APIs

• Type: New Feature
• Status: Triage Needed
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.18.0
• Component/s: dsl-sql
• Assignee: Kirill Kozlov
• Reporter: Kirill Kozlov
• Created: 23/Oct/19 20:43
• Updated: 13/Apr/23 10:58
• Resolved: 13/Nov/19 23:01

Description

• InMemoryTable should implement a following methods:

public PCollection<Row> buildIOReader(
    PBegin begin, BeamSqlTableFilter filters, List<String> fieldNames);

public BeamSqlTableFilter constructFilter(List<RexNode> filter);

• Update a push-down rule to support predicate/filter push-down.
• Create a class

class TestTableFilter implements BeamSqlTableFilter

 

Issue Links

• blocks: BEAM-8343 Add means for IO APIs to support predicate and/or project push-down when running SQL pipelines
• contains: BEAM-8508 [SQL] Support predicate push-down without project push-down
• links to: GitHub Pull Request #9863