[IMPALA-5170] Streaming gzip decompression for Parquet and Avro files

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: Impala 2.5.0, Impala 2.6.0, Impala 2.7.0, Impala 2.8.0
• Component/s: Backend
• Labels: ramp-up usability
• Epic Color: ghx-label-2
• Assignee: Unassigned
• Reporter: Alexander Behm
• Created: 05/Apr/17 18:52
• Updated: 22/Nov/17 19:56
• Resolved: 22/Nov/17 19:56

Description

To reduce the memory consumption of scans over gzip-compressed data, we should implement streaming decompression (gzip supports it).

Note that our text scanners already perform streaming decompression by default, but that's not the case for other scanners (e.g. Parquet/Avro).