[ORC-670] RecordReaderImpl.findColumns should respect orc.schema.evolution.case.sensitive

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.6.0, 1.6.1, 1.6.2, 1.6.3, 1.6.4, 1.6.5, 1.7.0
• Fix Version/s: 1.6.6, 1.7.0
• Component/s: Java
• Assignee: Dongjoon Hyun
• Reporter: Dongjoon Hyun
• Created: 05/Oct/20 06:12
• Updated: 10/Dec/20 23:36
• Resolved: 05/Oct/20 20:40

Description

This is a performance regression from 1.5.x. Apache ORC 1.6 always behaves case-sensitively for predicate pushdown.

Without this, ORC 1.6.x cannot pass SPARK-32622 (case sensitivity in predicate pushdown) unit test case.

Issue Links

• blocks: SPARK-33295 Upgrade ORC to 1.6.6
• links to: GitHub Pull Request #548