[SPARK-12126] JDBC datasource processes filters only commonly pushed down.

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Component/s: SQL
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Hyukjin Kwon
• Created: 03/Dec/15 11:07
• Updated: 12/Dec/22 18:10
• Resolved: 08/Oct/19 05:41

Description

As suggested here, Currently JDBC datasource only processes the filters pushed down from DataSourceStrategy.

Unlike ORC or Parquet, this can process pretty a lot of filters (for example, a + b > 3) since it is just about string parsing.

As here, using CatalystScan trait might be one of solutions.

Issue Links

• is duplicated by: SPARK-10899 Support JDBC pushdown for additional commands
• relates to: SPARK-12449 Pushing down arbitrary logical plans to data sources
• relates to: SPARK-12506 Push down WHERE clause arithmetic operator to JDBC layer
• links to: [Github] Pull Request #11005 (huaxingao)