[SPARK-27913] Spark SQL's native ORC reader implements its own schema evolution

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.3.3
• Component/s: SQL
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Owen O'Malley
• Created: 31/May/19 21:34
• Updated: 12/Dec/22 18:11
• Resolved: 25/May/21 01:38

Description

ORC's reader handles a wide range of schema evolution, but the Spark SQL native ORC bindings do not provide the desired schema to the ORC reader. This causes a regression when moving spark.sql.orc.impl from 'hive' to 'native'.