[ARROW-3646] [Python] Add convenience factories to create IO streams

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.11.0
• Fix Version/s: 0.12.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/19953
• Assignee: Antoine Pitrou
• Reporter: Antoine Pitrou
• Created: 29/Oct/18 18:09
• Updated: 11/Jan/23 07:28
• Resolved: 07/Nov/18 17:11

Description

Currently, creating IO streams requires invoking various constructors with irregular names. It would be nice to expose a higher-level interface.

For example:

def open_reader(source, compression='detect'):
    """
    Create an Arrow input stream.

    Parameters
    ----------
    source: str, Path, buffer, file-like object, ...
        The source to open for reading
    compression: str or None
        The compression algorithm to use for on-the-fly decompression.
        If 'detect' and source is a file path, then compression will be
        chosen based on the file extension.
        If None, no compression will be applied.
        Otherwise, a well-known algorithm name must be supplied (e.g. "gzip")
    """

def open_writer(source, compression='detect'):
    """
    Create an Arrow output stream.

    Parameters
    ----------
    source: str, Path, buffer, file-like object, ...
        The source to open for writing
    compression: str or None
        The compression algorithm to use for on-the-fly compression.
        If 'detect' and source is a file path, then compression will be
        chosen based on the file extension.
        If None, no compression will be applied.
        Otherwise, a well-known algorithm name must be supplied (e.g. "gzip")
    """

Thoughts?

Issue Links

• supercedes: ARROW-3645 [Python] Document compression support in Sphinx
• links to: GitHub Pull Request #2873