[SPARK-26427] Upgrade Apache ORC to 1.5.4

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 2.4.1, 3.0.0
• Component/s: Build
• Assignee: Dongjoon Hyun
• Reporter: Dongjoon Hyun
• Created: 21/Dec/18 17:58
• Updated: 27/Jan/19 01:21
• Resolved: 22/Dec/18 08:42

Description

This issue aims to update Apache ORC dependency to the latest version 1.5.4 released at Dec. 20. (Release Notes)

[ORC-237] - OrcFile.mergeFiles Specified block size is less than configured minimum value
[ORC-409] - Changes for extending MemoryManagerImpl
[ORC-410] - Fix a locale-dependent test in TestCsvReader
[ORC-416] - Avoid opening data reader when there is no stripe
[ORC-417] - Use dynamic Apache Maven mirror link
[ORC-419] - Ensure to call `close` at RecordReaderImpl constructor exception
[ORC-432] - openjdk 8 has a bug that prevents surefire from working
[ORC-435] - Ability to read stripes that are greater than 2GB
[ORC-437] - Make acid schema checks case insensitive
[ORC-411] - Update build to work with Java 10.
[ORC-418] - Fix broken docker build script

Issue Links

• blocks: SPARK-20901 Feature parity for ORC with Parquet
• is related to: SPARK-23458 Flaky test: OrcQuerySuite
• relates to: SPARK-24417 Build and Run Spark on JDK11
• links to: GitHub Pull Request #23364
• links to: GitHub Pull Request #23646