[SPARK-22386] Data Source V2 improvements

• Type: Improvement
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.3.0
• Component/s: SQL
• Labels: releasenotes
• Target Version/s: 3.0.0
• Assignee: Unassigned
• Reporter: Wenchen Fan
• Created: 29/Oct/17 12:53
• Updated: 16/Nov/18 14:16

Issue Links

• depends upon: SPARK-25531 new write APIs for data source v2
• is related to: SPARK-23521 SPIP: Standardize SQL logical plans with DataSourceV2
• relates to: SPARK-26088 DataSourceV2 should expose row count and attribute statistics

Sub-Tasks

• Limit push down: Open
• Aggregate push down: Open
• Data Source V2: Join Push Down: Open
• Add support for USING syntax for DataSourceV2: In Progress
• DataSourceV2: Add interfaces to pass required sorting and clustering for writes: In Progress
• Recover options and properties and pass them back into the v1 API: Open