[ARROW-16119] [Python] Deprecate the legacy ParquetDataset custom python-based implementation

• Type: Task
• Status: Reopened
• Priority: Major
• Resolution: Unresolved
• Component/s: Python
• External issue URL: https://github.com/apache/arrow/issues/31529
• Assignee: Unassigned
• Reporter: Joris Van den Bossche
• Created: 05/Apr/22 11:25
• Updated: 11/Jan/23 11:41

Description

To be able to remove the custom python implementation (ARROW-15868), we first need to deprecate the various aspects.

This issue is meant as a parent issue to keep an overview of the different tasks.

Issue Links

• blocks: ARROW-15868 [Python] Remove the legacy ParquetDataset custom python-based implementation
• is a child of: ARROW-9720 [Python] Long-term fate of pyarrow.parquet.ParquetDataset
• is required by: ARROW-16481 [Python] Remove the test usage of legacy dataset

Sub-Tasks

• [Python] Start with deprecating ParquetDataset custom attributes: Resolved
• [Python] Start to raise deprecation warnings when using use_legacy_dataset=True in parquet.py: Resolved
• [Python] Start raising deprecation warnings for ParquetDataset keywords that won't be supported with the new API: Resolved
• [Python] ParquetDataset deprecation: change Deprecation to FutureWarnings: Resolved
• [Python] Deprecate the (common_)metadata(_path) attributes of ParquetDataset: Resolved
• [Python] Change use_legacy_dataset default and deprecate no-longer supported keywords in parquet.write_to_dataset: Resolved
• [Python] Support row_group_size/chunk_size keyword in pq.write_to_dataset with use_legacy_dataset=False: Resolved
• [Python] Switch default and deprecate use_legacy_dataset=True in ParquetDataset: Resolved