[ARROW-10574] [Python][Parquet] Allow collections for 'in' / 'not in' filter (in addition to sets)

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/26538
• Assignee: Weiyang Zhao
• Reporter: Weiyang Zhao
• Created: 13/Nov/20 00:45
• Updated: 11/Jan/23 08:14
• Resolved: 21/Dec/20 16:44

Description

I would like to enhance partition filters in methods such as:

pyarrow.parquet.ParquetDataset(path, filters)

I am proposing the below enhancements:

1. for operator "in", "not in", the value should be any typing.Iteratable (also a container). But currently only set is supported while other iteratable, such as list, tuple cannot function correctly. I would like to change it to accept any iteratable.
2. Enhance the documents about the partition filters.

I see there is a new version implemented with 
_ParquetDatasetV2 which already accepts an iterable. So the documentation update is fine for the new version as well.
 

Issue Links

• links to: GitHub Pull Request #8672