[ARROW-16122] [Python] Change use_legacy_dataset default and deprecate no-longer supported keywords in parquet.write_to_dataset

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 8.0.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/31531
• Assignee: Alenka Frim
• Reporter: Joris Van den Bossche
• Created: 05/Apr/22 11:49
• Updated: 11/Jan/23 11:42
• Resolved: 21/Apr/22 07:49

Description

Currently, the pq.write_to_dataset function also had a use_legacy_dataset keyword, but we should:

1) in case of use_legacy_dataset=True, ensure we raise deprecation warnings for all keywords that won't be supported in the new implementation (eg partition_filename_cb)
2) raise a deprecation warning for use_legacy_dataset=True, and/or already switch the default?

Issue Links

• links to: GitHub Pull Request #12811