P2 Description
The objective is to create a universal way for Beam SQL IO APIs to support predicate/project push-down.
A proposed way to achieve that is by introducing an interface responsible for identifying what portion(s) of a Calc can be moved down to IO layer. Also, adding following methods to a BeamSqlTable interface to pass necessary parameters to IO APIs:
- BeamSqlTableFilter constructFilter(List<RexNode> filter)
- ProjectSupport supportsProjects()
- PCollection<Row> buildIOReader(PBegin begin, BeamSqlTableFilter filters, List<String> fieldNames)
ProjectSupport is an enum with the following options:
- NONE
- WITHOUT_FIELD_REORDERING
- WITH_FIELD_REORDERING
Design doc link.
