[SPARK-30627] Disable all the V2 file sources in Spark 3.0 by default

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: SQL
• Assignee: Gengliang Wang
• Reporter: Gengliang Wang
• Created: 23/Jan/20 23:53
• Updated: 24/Jan/20 08:08
• Resolved: 24/Jan/20 05:43

Description

There are still some missing parts in the file source V2 framework:
1. It doesn't support reporting file scan metrics such as "numOutputRows"/"numFiles"/"fileSize" like `FileSourceScanExec`.
2. It doesn't support partition pruning with subqueries or dynamic partition pruning.

As we are going to code freeze on Jan 31st, I suggest disabling all the V2 file sources in Spark 3.0 by default.

Issue Links

• links to: GitHub Pull Request #27348