[ARROW-10425] [Python] Support reading (compressed) CSV file from remote file / binary blob

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Python
• Labels: csv
• External issue URL: https://github.com/apache/arrow/issues/26404
• Assignee: Unassigned
• Reporter: Joris Van den Bossche
• Created: 29/Oct/20 20:32
• Updated: 11/Jan/23 08:13

Description

From https://stackoverflow.com/questions/64588076/how-can-i-read-a-csv-gz-file-with-pyarrow-from-a-file-object

Currently pyarrow.csv.rad_csv happily takes a path to a compressed file and automatically decompresses it, but AFAIK this only works for local paths.

It would be nice to in general support reading CSV from remote files (with URI / specifying a filesystem), and in that case also support compression.

In addition we could also read a compressed file from a BytesIO / file-like object, but not sure we want that (as it would required a keyword to indicate the used compression).