[ARROW-8651] [Python][Dataset] Support pickling of Dataset objects

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Component/s: Python
• Labels: dataset pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/24812
• Assignee: Krisztian Szucs
• Reporter: Joris Van den Bossche
• Created: 30/Apr/20 15:36
• Updated: 11/Jan/23 08:01
• Resolved: 07/Jul/20 08:21

Description

We alraedy made several parts of a Dataset serializable (the formats, the expressions, the filesystem). With those, it should also be possible to pickle FileFragments, and with that also Dataset.

Issue Links

• relates to: ARROW-9332 [Python][Dataset] Support pickling of ParquetFileFragment's RowGroupInfo
• links to: GitHub Pull Request #7631