[SPARK-41858] Fix ORC reader perf regression due to DEFAULT value feature

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: SQL
• Target Version/s: 3.4.0
• Assignee: Dongjoon Hyun
• Reporter: Dongjoon Hyun
• Created: 03/Jan/23 09:48
• Updated: 03/Jan/23 18:45
• Resolved: 03/Jan/23 18:40

Description

A huge ORC reader perf regression is detected by SPARK-41782. The root cause was SPARK-39862.

Issue Links

• is caused by: SPARK-39862 Fix bug in existence DEFAULT value lookups for V2 data sources
• is related to: SPARK-41862 Fix a correctness bug in existence DEFAULT value lookups for the Orc data source
• links to: [Github] Pull Request #39362 (dongjoon-hyun)