[ARROW-9749] [C++][Dataset] Extract format-specific scan options from FileFormat

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 4.0.0
• Component/s: C++
• Labels: dataset pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/25801
• Assignee: David Li
• Reporter: Ben Kietzman
• Created: 14/Aug/20 21:46
• Updated: 11/Jan/23 08:08
• Resolved: 16/Mar/21 13:21

Description

Currently format specific scan options are embedded as members of the corresponding subclass of FileFormat. Extracting these to an options struct would provide better separation of concerns; currently the only way to scan a parquet formatted dataset with different options is to reconstruct it in a differently optioned format from its component files.

CsvFileFormat could retain ParseOptions as a member, since (for example) tab-separated vs comma-separated values can justifiably be considered different formats.

Issue Links

• is related to: ARROW-11974 [GLib] Add CsvFragmentScanOption support
• is related to: ARROW-12059 [R] Accept format-specific scan options in collect()
• is related to: ARROW-11972 [C++][Dataset] Extract IpcFragmentScanOptions, ParquetFragmentScanOptions
• relates to: ARROW-8631 [C++][Dataset] Add ConvertOptions and ReadOptions to CsvFileFormat
• links to: GitHub Pull Request #9686