[IMPALA-1506] Gzip text compressor doesn't always set output_length

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.0
• Fix Version/s: Impala 2.1
• Assignee: Victor Bittorf
• Reporter: Matthew Jacobs
• Created: 21/Nov/14 21:26
• Updated: 24/Nov/14 19:13
• Resolved: 24/Nov/14 19:13

Description

We hit a DCHECK failure in Compress() because output_length isn't always set. We need to set the output_length to the real output buffer size or zlib won't make any progress (or return with an error).

create table temp like tpch_text_gzip.lineitem ;
set compression_codec=GZIP;
set ALLOW_UNSUPPORTED_FORMATS=1;
insert into temp select * from tpch_text_gzip.lineitem;

I think the fix is trivial (see below), but we should add a test case for this too.

diff --git a/be/src/util/compress.cc b/be/src/util/compress.cc
index 3713626..d0f57e4 100644
--- a/be/src/util/compress.cc
+++ b/be/src/util/compress.cc
@@ -118,6 +118,7 @@ Status GzipCompressor::ProcessBlock(bool output_preallocated,
       *output_length = buffer_length_;
     }
     *output = out_buffer_;
+    *output_length = buffer_length_;
   } else if (*output_length < max_compressed_len) {
     return Status("GzipCompressor::ProcessBlock: output length too small");
   }