Description
Discussion about adding TableCatalog on the dev list focused around whether Expression should be used in the public DataSourceV2 API, with SupportsPushDownCatalystFilters as an example of where it is already exposed. The early consensus is that Expression should not be exposed in the public API.
From rxin:
I completely disagree with using Expression in critical public APIs that we expect a lot of developers to use . . . If we are depending on Expressions on the more common APIs in dsv2 already, we should revisit that.
The main use of this API is to pass Expression to FileFormat classes that used Expression instead of Filter. External sources also use it for more complex push-down, like to_date(ts) = '2018-05-13', but those uses can be done with Analyzer rules or when translating to Filters.
Attachments
Issue Links
- is duplicated by
-
SPARK-25192 Remove SupportsPushdownCatalystFilter
-
- Resolved
-
- links to
