[ARROW-2882] [C++][Python] Support AWS Firehose partition_scheme implementation for Parquet datasets

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.17.0
• Component/s: C++, Python
• Labels: dataset dataset-parquet-read parquet
• External issue URL: https://github.com/apache/arrow/issues/19253
• Assignee: Unassigned
• Reporter: Pablo Javier Takara
• Created: 19/Jul/18 13:48
• Updated: 11/Jan/23 07:23
• Resolved: 16/Jun/20 06:10

Description

I'd like to be able to read a ParquetDataset generated by AWS Firehose.

The only implementation at the time of writting was the partition scheme created by hive (year=2018/month=01/day=11).

AWS Firehose partition scheme is a little bit different (2018/01/11).

 

Thanks

Issue Links

• depends upon: ARROW-8039 [Python][Dataset] Support using dataset API in pyarrow.parquet with a minimal ParquetDataset shim