[ARROW-9657] [R][Dataset] Expose more FileSystemDatasetFactory options

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: R
• Labels: dataset
• External issue URL: https://github.com/apache/arrow/issues/25716
• Assignee: Unassigned
• Reporter: Neal Richardson
• Created: 05/Aug/20 18:00
• Updated: 11/Jan/23 08:08

Description

Among the features:

• ignore_prefixes option
• Pass an explicit list of files + base directory
• Exclude invalid files (boolean) option

An important use case this would allow/fix is being able to open_dataset("really_really_big_file.csv") so you can partition/write it.

Issue Links

• is duplicated by: ARROW-12036 [R] dataset by a single parquet file
• is related to: ARROW-9748 [C++][Dataset] Remove Selector, ignore_prefixes from FileSystemDatasetFactory
• relates to: ARROW-11789 [R] Support vector of multiple file paths in read_*() functions

Sub-Tasks

• [R][Dataset] Allow create dataset from vector of file paths: Resolved