[ORC-416] Avoid opening data reader when there is no stripe

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.0, 1.5.1, 1.5.2, 1.5.3
• Fix Version/s: 1.5.4, 1.6.0
• Component/s: Java
• Assignee: Dongjoon Hyun
• Reporter: Dongjoon Hyun
• Created: 10/Oct/18 09:35
• Updated: 02/Jan/19 19:12
• Resolved: 10/Oct/18 17:02

Description

Currently, `RecordReaderImpl` invokes `dataReader.open` in the middle of the constructor. We can postpone this until `advanceToNextRow` reads the stripe at the end of the constructor.

this.dataReader.open();

This will reduce the chance of potential open-file leakages due to IOException and OOM during `RecordReaderImpl` construction. Also, we can avoid it completely when there is no stripe in the file.

Issue Links

• is related to: SPARK-23390 Flaky test: FileBasedDataSourceSuite
• is related to: SPARK-25688 Potential resource leak in ORC
• links to: GitHub Pull Request #320