[ORC-319] C++ reader crash after Lz4DecompressionStream thrown a ParseError

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Resolved
• Affects Version/s: 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.4.0, 1.4.1, 1.4.2, 1.4.3
• Fix Version/s: 1.5.0
• Component/s: C++
• Assignee: Quanlong Huang
• Reporter: Quanlong Huang
• Created: 08/Mar/18 12:13
• Updated: 14/May/18 22:24
• Resolved: 16/Mar/18 01:47

Description

The c++ reader crashes on reading a corrupt file (see attachments).

terminate called after throwing an instance of 'orc::ParseError*’

It crashes after throwing a ParseError exception from orc::Lz4DecompressionStream::decompress

(gdb) bt
#0  0x00007ffff6a12c37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007ffff6a16028 in __GI_abort () at abort.c:89
#2  0x00007ffff7321535 in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007ffff731f6d6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007ffff731f703 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007ffff731f922 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x000000000089ca24 in orc::Lz4DecompressionStream::decompress (this=0xf4db40, 
    input=0xf53f68 "\343`\026x\305)\305\307\301,\360\200I\242\221M\241\215Qc\033\223R,\a\217\020\067#\023\063\v+\033;\a'\027\267\024k|r~\216\001\204\062\204PF\020\312\030B\231@(S\be\006\241\314!\224\005\204\262\224b\003k7Pb\342`\006b\006 f\004b&(\237\005\210Y\201\230\r\210١\230\323`\033\223\025+\307\066\246", length=270, output=0x7ffff7f60010 "`\026x\305)\305\307\301,\360\200I\242\221M\241", 
    maxOutputLength=262144) at /mnt/volume1/orc/c++/src/Compression.cc:882
#7  0x000000000089c3c1 in orc::BlockDecompressionStream::Next (this=0xf4db40, data=0x7fffffffe0f8, size=0x7fffffffe0f4) at /mnt/volume1/orc/c++/src/Compression.cc:722
#8  0x00000000008ac4ba in NextNonEmpty (size=0x7fffffffe0f4, data=0x7fffffffe0f8, input=0xf4db40) at google/protobuf/io/coded_stream.cc:63
#9  google::protobuf::io::CodedInputStream::Refresh (this=this@entry=0x7fffffffe140) at google/protobuf/io/coded_stream.cc:551
#10 0x00000000008aa06c in CodedInputStream (input=<optimized out>, this=0x7fffffffe140) at ./google/protobuf/io/coded_stream.h:1183
#11 google::protobuf::MessageLite::ParseFromZeroCopyStream (this=0xf57d10, input=<optimized out>) at google/protobuf/message_lite.cc:169
#12 0x0000000000834541 in orc::readFooter (stream=0xf51680, buffer=@0x7fffffffe2b0: 0xf4f450, footerOffset=1429, ps=..., memoryPool=...) at /mnt/volume1/orc/c++/src/Reader.cc:925
#13 0x0000000000834b57 in orc::createReader (stream=std::unique_ptr<orc::InputStream> containing 0xf51680, options=...) at /mnt/volume1/orc/c++/src/Reader.cc:978
#14 0x000000000082d28c in scanFile (out=..., filename=0x7fffffffe776 "decompress_crash.orc", batchSize=1024) at /mnt/volume1/orc/tools/src/FileScan.cc:32
#15 0x000000000082d5a5 in main (argc=1, argv=0x7fffffffe510) at /mnt/volume1/orc/tools/src/FileScan.cc:84
(gdb) f 6
#6  0x000000000089ca24 in orc::Lz4DecompressionStream::decompress (this=0xf4db40, 
    input=0xf53f68 "\343`\026x\305)\305\307\301,\360\200I\242\221M\241\215Qc\033\223R,\a\217\020\067#\023\063\v+\033;\a'\027\267\024k|r~\216\001\204\062\204PF\020\312\030B\231@(S\be\006\241\314!\224\005\204\262\224b\003k7Pb\342`\006b\006 f\004b&(\237\005\210Y\201\230\r\210١\230\323`\033\223\025+\307\066\246", length=270, output=0x7ffff7f60010 "`\026x\305)\305\307\301,\360\200I\242\221M\241", 
    maxOutputLength=262144) at /mnt/volume1/orc/c++/src/Compression.cc:882
882           throw new ParseError(getName() + " - failed to decompress");
(gdb) l
877                                                   char *output,
878                                                   size_t maxOutputLength) {
879         int result = LZ4_decompress_safe(input, output, static_cast<int>(length),
880                                          static_cast<int>(maxOutputLength));
881         if (result < 0) {
882           throw new ParseError(getName() + " - failed to decompress");
883         }
884         return static_cast<uint64_t>(result);
885       }
886

Issue Links

• blocks: ORC-315 More robust on reading random corrupt files
• links to: GitHub Pull Request #228