[IMPALA-7058] RC and Seq fuzz tests cause crash

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 3.0
• Fix Version/s: Impala 2.13.0, Impala 3.1.0
• Component/s: Backend
• Labels: broken-build crash
• Epic Color: ghx-label-5
• Assignee: Tim Armstrong
• Reporter: Dimitris Tsirogiannis
• Created: 22/May/18 22:05
• Updated: 02/Jun/18 19:36
• Resolved: 23/May/18 20:47

Description

The backtrace is here:

#7 0x0000000002d89a84 in impala::DelimitedTextParser<false>::ParseFieldLocations (this=0xcf539a0, max_tuples=1, remaining_len=-102, byte_buffer_ptr=0x7fc6b764dad0, row_end_locations=0x7fc6b764dac0, field_locations=0x10034000, num_tuples=0x7fc6b764dacc, num_fields=0x7fc6b764dac8, next_column_start=0x7fc6b764dad8) at /data/jenkins/workspace/impala-asf-master-exhaustive-rhel7/repos/Impala/be/src/exec/delimited-text-parser.cc:205
#8 0x0000000001fdb641 in impala::HdfsSequenceScanner::ProcessRange (this=0x15515f80, row_batch=0xcf54800) at /data/jenkins/workspace/impala-asf-master-exhaustive-rhel7/repos/Impala/be/src/exec/hdfs-sequence-scanner.cc:352
#9 0x0000000002d7a20e in impala::BaseSequenceScanner::GetNextInternal (this=0x15515f80, row_batch=0xcf54800) at /data/jenkins/workspace/impala-asf-master-exhaustive-rhel7/repos/Impala/be/src/exec/base-sequence-scanner.cc:181
#10 0x0000000001fb1ff0 in impala::HdfsScanner::ProcessSplit (this=0x15515f80) at /data/jenkins/workspace/impala-asf-master-exhaustive-rhel7/repos/Impala/be/src/exec/hdfs-scanner.cc:134
#11 0x0000000001f89258 in impala::HdfsScanNode::ProcessSplit (this=0x2a4a8800, filter_ctxs=..., expr_results_pool=0x7fc6b764e4b0, scan_range=0x13f5f8700, scanner_thread_reservation=0x7fc6b764e428) at /data/jenkins/workspace/impala-asf-master-exhaustive-rhel7/repos/Impala/be/src/exec/hdfs-scan-node.cc:453
#12 0x0000000001f885f9 in impala::HdfsScanNode::ScannerThread (this=0x2a4a8800, first_thread=false, scanner_thread_reservation=32768) at /data/jenkins/workspace/impala-asf-master-exhaustive-rhel7/repos/Impala/be/src/exec/hdfs-scan-node.cc:360
#13 0x0000000001f87a6c in impala::HdfsScanNode::<lambda()>::operator()(void) const (__closure=0x7fc6b764ebe8) at /data/jenkins/workspace/impala-asf-master-exhaustive-rhel7/repos/Impala/be/src/exec/hdfs-scan-node.cc:292
#14 0x0000000001f89ac8 in boost::detail::function::void_function_obj_invoker0<impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)::<lambda()>, void>::invoke(boost::detail::function::function_buffer &) (function_obj_ptr=...) at /data/jenkins/workspace/impala-asf-master-exhaustive-rhel7/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153
#15 0x0000000001bf0b28 in boost::function0<void>::operator() (this=0x7fc6b764ebe0) at /data/jenkins/workspace/impala-asf-master-exhaustive-rhel7/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
#16 0x0000000001edc57f in impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long>*) (name=..., category=..., functor=..., parent_thread_info=0x7fc6b9e53890, thread_started=0x7fc6b9e527c0) at /data/jenkins/workspace/impala-asf-master-exhaustive-rhel7/repos/Impala/be/src/util/thread.cc:356
#17 0x0000000001ee471b in boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long>*> >::operator()<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long>*), boost::_bi::list0&, int) (this=0x2a370fc0, f=@0x2a370fb8: 0x1edc218 <impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long>*)>, a=...) at /data/jenkins/workspace/impala-asf-master-exhaustive-rhel7/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525
#18 0x0000000001ee463f in boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long>*> > >::operator()() (this=0x2a370fb8) at /data/jenkins/workspace/impala-asf-master-exhaustive-rhel7/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20
#19 0x0000000001ee4602 in boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long>*> > > >::run() (this=0x2a370e00) at /data/jenkins/workspace/impala-asf-master-exhaustive-rhel7/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116
#20 0x00000000031e1c5a in thread_proxy ()
#21 0x00007fc76c44ddf3 in start_thread () from /lib64/libpthread.so.0
#22 0x00007fc76c17b1ad in clone () from /lib64/libc.so.6

Issue Links

• is broken by: IMPALA-3833 Fix invalid data handling in Sequence and RCFile scanners
• is duplicated by: IMPALA-7113 ASAN heap-buffer-overflow in impala::HdfsRCFileScanner::GetCurrentKeyBuffer()
• is related to: IMPALA-7065 Fix remaining invalid data handling bugs in RC and sequence file