[ARROW-9682] [Python] Unable to specify the partition style with pq.write_to_dataset

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 1.0.0
• Component/s: Python
• Labels: dataset-parquet-write parquet parquetWriter
• Environment: Ubuntu 18.04 ; ; Python 3.7
• External issue URL: https://github.com/apache/arrow/issues/25738
• Assignee: Unassigned
• Reporter: Lance Dacey
• Created: 10/Aug/20 15:42
• Updated: 11/Jan/23 08:08
• Resolved: 16/Apr/21 10:13

Description

I am able to import and test DirectoryPartitioning but I am not able to figure out a way to write a dataset using this feature. It seems like write_to_dataset defaults to the "hive" style. Is there a way to test this?

from pyarrow.dataset import DirectoryPartitioning

partitioning = DirectoryPartitioning(pa.schema([("year", pa.int16()), ("month", pa.int8()), ("day", pa.int8())]))

print(partitioning.parse("/2009/11/3"))