[ARROW-11381] [Rust] [Parquet] LZ4 compressed files written in Rust can't be opened with C++

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.0
• Component/s: Rust
• External issue URL: https://github.com/apache/arrow/issues/27274
• Assignee: Unassigned
• Reporter: Neville Dipale
• Created: 25/Jan/21 18:19
• Updated: 11/Jan/23 08:19

Description

Parquet files that are written with LZ4 compression, cannot be read from pyarrow. It seems that the issue might be the LZ4 block vs frame, which we're also seeing in ARROW-8767.

I'll update this JIRA with more info, as I'm struggling to get pyspark up on MacOS (Rosetta 2 issues)