[ORC-590] Crash in orc::RleDecoderV2::readByte

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.7.0
• Component/s: C++
• Assignee: Zoltán Borók-Nagy
• Reporter: Quanlong Huang
• Created: 15/Jan/20 23:45
• Updated: 21/Sep/21 20:44
• Resolved: 20/Jan/20 13:39

Description

Hit a crash when reading a corrupt file.

(gdb) bt
#0  0x00000000006108ad in orc::RleDecoderV2::readByte (this=0xd5a0d0) at /home/quanlong/workspace/orc/c++/src/RLEv2.hh:167
#1  orc::RleDecoderV2::next (this=0xd5a0d0, data=0xd5a1d8, numValues=30, notNull=0x0) at /home/quanlong/workspace/orc/c++/src/RleDecoderV2.cc:119
#2  0x00000000005f6b8c in orc::StringDictionaryColumnReader::StringDictionaryColumnReader (this=this@entry=0xb497a0, type=..., stripe=...) at /home/quanlong/workspace/orc/c++/src/ColumnReader.cc:581
#3  0x00000000005f70bb in orc::buildReader (type=..., stripe=...) at /home/quanlong/workspace/orc/c++/src/ColumnReader.cc:1756
#4  0x00000000005f722b in orc::StructColumnReader::StructColumnReader (this=this@entry=0xb07e40, type=..., stripe=...) at /home/quanlong/workspace/orc/c++/src/ColumnReader.cc:876
#5  0x00000000005f701b in orc::buildReader (type=..., stripe=...) at /home/quanlong/workspace/orc/c++/src/ColumnReader.cc:1787
#6  0x000000000059fd18 in orc::RowReaderImpl::startNextStripe (this=0xae2750) at /home/quanlong/workspace/orc/c++/src/Reader.cc:917
#7  0x00000000005a016a in orc::RowReaderImpl::next (this=0xae2750, data=...) at /home/quanlong/workspace/orc/c++/src/Reader.cc:932
#8  0x0000000000597a78 in scanFile (out=..., filename=<optimized out>, batchSize=batchSize@entry=1024) at /home/quanlong/workspace/orc/tools/src/FileScan.cc:39
#9  0x00000000005972f8 in main (argc=1, argv=<optimized out>) at /home/quanlong/workspace/orc/tools/src/FileScan.cc:84
(gdb) l
162	
163	  unsigned char readByte() {
164	  if (bufferStart == bufferEnd) {
165	    int bufferLength;
166	    const void* bufferPointer;
167	    if (!inputStream->Next(&bufferPointer, &bufferLength)) {
168	      throw ParseError("bad read in RleDecoderV2::readByte");
169	    }
170	    bufferStart = static_cast<const char*>(bufferPointer);
171	    bufferEnd = bufferStart + bufferLength;

Issue Links

• causes: IMPALA-9297 HdfsOrcScanner crash in orc::RleDecoderV2::next
• is caused by: ORC-469 Add EncodedStringVectorBatch to expose string dictionary in VectorBatch
• links to: GitHub Pull Request #469