[ARROW-14612] [C++] Support for filename-based partitioning

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 8.0.0
• Component/s: C++
• Labels: dataset good-second-issue pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/30158
• Assignee: Sanjiban Sengupta
• Reporter: Nicola Crane
• Created: 05/Nov/21 16:42
• Updated: 11/Jan/23 08:41
• Resolved: 30/Mar/22 13:09

Description

Directory-based partitioning is a feature of Arrow, but could we support filename-based partitioning?

e.g. I have a series of CSV files here all called something like foo_month_year.csv and it'd be nice to be able to read them in and then the month/year bits of the filenames then appear as fields I can filter on etc.

 

Issue Links

• is related to: ARROW-15260 [R] open_dataset - add file_name as column
• is related to: ARROW-16080 [R][Documentation] Document filename-based partitioning and filename-as-variable functionality
• is related to: ARROW-15281 [C++] Implement ability to retrieve fragment filename
• relates to: ARROW-16113 [Python] Partitioning.dictionaries in case of a subset of fields are dictionary encoded
• relates to: ARROW-14176 [Python] Filename-based partitioning scheme
• links to: GitHub Pull Request #12530