[ARROW-2677] [Python] Expose Parquet ZSTD compression

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9.0
• Fix Version/s: 0.10.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/19069
• Assignee: Uwe Korn
• Reporter: William Lee
• Created: 06/Jun/18 14:41
• Updated: 11/Jan/23 07:22
• Resolved: 14/Jun/18 16:24

Description

It seems like in the python/pyarrow/_parquet.pyx file there's a check for the supported compression scheme:

cdef int check_compression_name(name) except -1:
    if name.upper() not in ['NONE', 'SNAPPY', 'GZIP', 'LZO', 'BROTLI', 'LZ4']:
        raise ArrowException("Unsupported compression: " + name)
    return 0

which does not include ZSTD (Zstandard).  From my understanding it should be supported in the underlying c++ library already.  Is it possible to add that support?

 

Issue Links

• links to: GitHub Pull Request #2120