[SPARK-11955] Mark one side fields in merging schema for safely pushdowning filters in parquet

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: SQL
• Assignee: L. C. Hsieh
• Reporter: L. C. Hsieh
• Created: 24/Nov/15 17:20
• Updated: 29/Jan/16 00:25
• Resolved: 29/Jan/16 00:25

Description

Currently we simply skip pushdowning filters in parquet if we enable schema merging.

However, we can actually mark one side fields in merging schema for safely pushdowning filters in parquet.

Issue Links

• links to: [Github] Pull Request #9940 (viirya)