[ARROW-6282] [Format] Support lossy compression

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Abandoned
• Fix Version/s: 2.0.0
• Component/s: Format
• External issue URL: https://github.com/apache/arrow/issues/22666
• Assignee: Unassigned
• Reporter: Dominik Moritz
• Created: 17/Aug/19 10:32
• Updated: 11/Jan/23 07:46
• Resolved: 07/Sep/20 16:35

Description

Arrow dataframes with large columns of integers or floats can be compressed using gzip or brotli. However, in some cases it will be okay to compress the data lossy to achieve even higher compression ratios. The main use case for this is visualization where small inaccuracies matter less.

Issue Links

• relates to: PARQUET-1716 [C++] Add support for BYTE_STREAM_SPLIT encoding