[IMPALA-7883] TestScannersFuzzing::test_fuzz_decimal_tbl generates crash in ParquetMetadataUtils::ValidateRowGroupColumn

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Cannot Reproduce
• Affects Version/s: Impala 3.2.0
• Component/s: Backend
• Labels: broken-build
• Target Version: Impala 4.0.0
• Epic Color: ghx-label-4
• Assignee: Csaba Ringhofer
• Reporter: Joe McDonnell
• Created: 22/Nov/18 20:14
• Updated: 17/Nov/20 20:21
• Resolved: 17/Nov/20 20:21

Description

An exhaustive test run ran into a SIGSEGV when running TestScannersFuzzing::test_fuzz_decimal_tbl(). It produces the following stack:

#0 0x00007f0bda92f1f7 in raise () from /lib64/libc.so.6
#1 0x00007f0bda9308e8 in abort () from /lib64/libc.so.6
#2 0x00007f0bddb00185 in os::abort(bool) () from /usr/java/jdk1.8.0_144/jre/lib/amd64/server/libjvm.so
#3 0x00007f0bddca2593 in VMError::report_and_die() () from /usr/java/jdk1.8.0_144/jre/lib/amd64/server/libjvm.so
#4 0x00007f0bddb0568f in JVM_handle_linux_signal () from /usr/java/jdk1.8.0_144/jre/lib/amd64/server/libjvm.so
#5 0x00007f0bddafbbe3 in signalHandler(int, siginfo*, void*) () from /usr/java/jdk1.8.0_144/jre/lib/amd64/server/libjvm.so
#6 <signal handler called>
#7 0x0000000002f0d254 in impala::ParquetMetadataUtils::ValidateRowGroupColumn (file_metadata=..., filename=0x1dab5d5b8 "hdfs://localhost:20500/test-warehouse/test_fuzz_decimal_tbl_4a8e12be.db/decimal_tbl/d6=1/copy2_6a476efdb58955a1-fcfa2ac500000000_1388116616_data.0.parq", row_group_idx=0, col_idx=3, schema_element=..., state=0x4fc1dee0) at /data/jenkins/workspace/impala-cdh6.1.0-exhaustive/repos/Impala/be/src/exec/parquet-metadata-utils.cc:156
#8 0x00000000022ad8c3 in impala::BaseScalarColumnReader::Reset (this=0x22877800, file_desc=..., col_chunk=..., row_group_idx=0) at /data/jenkins/workspace/impala-cdh6.1.0-exhaustive/repos/Impala/be/src/exec/parquet-column-readers.cc:928
#9 0x00000000022357b0 in impala::HdfsParquetScanner::InitScalarColumns (this=0x22608800) at /data/jenkins/workspace/impala-cdh6.1.0-exhaustive/repos/Impala/be/src/exec/hdfs-parquet-scanner.cc:1501
#10 0x000000000222d60f in impala::HdfsParquetScanner::NextRowGroup (this=0x22608800) at /data/jenkins/workspace/impala-cdh6.1.0-exhaustive/repos/Impala/be/src/exec/hdfs-parquet-scanner.cc:626
#11 0x000000000222bd26 in impala::HdfsParquetScanner::GetNextInternal (this=0x22608800, row_batch=0x505cbbc0) at /data/jenkins/workspace/impala-cdh6.1.0-exhaustive/repos/Impala/be/src/exec/hdfs-parquet-scanner.cc:419
#12 0x000000000222a249 in impala::HdfsParquetScanner::ProcessSplit (this=0x22608800) at /data/jenkins/workspace/impala-cdh6.1.0-exhaustive/repos/Impala/be/src/exec/hdfs-parquet-scanner.cc:336
#13 0x00000000021ae94a in impala::HdfsScanNode::ProcessSplit (this=0x156b7000, filter_ctxs=..., expr_results_pool=0x7f0b22a61420, scan_range=0x22f64140, scanner_thread_reservation=0x7f0b22a61378) at /data/jenkins/workspace/impala-cdh6.1.0-exhaustive/repos/Impala/be/src/exec/hdfs-scan-node.cc:497
#14 0x00000000021adb97 in impala::HdfsScanNode::ScannerThread (this=0x156b7000, first_thread=true, scanner_thread_reservation=40960) at /data/jenkins/workspace/impala-cdh6.1.0-exhaustive/repos/Impala/be/src/exec/hdfs-scan-node.cc:402
#15 0x00000000021acf88 in impala::HdfsScanNode::<lambda()>::operator()(void) const (__closure=0x7f0b22a61ba8) at /data/jenkins/workspace/impala-cdh6.1.0-exhaustive/repos/Impala/be/src/exec/hdfs-scan-node.cc:323
#16 0x00000000021af426 in boost::detail::function::void_function_obj_invoker0<impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)::<lambda()>, void>::invoke(boost::detail::function::function_buffer &) (function_obj_ptr=...) at /data/jenkins/workspace/impala-cdh6.1.0-exhaustive/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153
#17 0x0000000001cc7614 in boost::function0<void>::operator() (this=0x7f0b22a61ba0) at /data/jenkins/workspace/impala-cdh6.1.0-exhaustive/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
#18 0x00000000020fe391 in impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*) (name=..., category=..., functor=..., parent_thread_info=0x7f0b2025c850, thread_started=0x7f0b2025b220) at /data/jenkins/workspace/impala-cdh6.1.0-exhaustive/repos/Impala/be/src/util/thread.cc:359

This reproduces fairly regularly (but not 100% consistently) on master when running:

export SCANNER_FUZZ_SEED=1542888792
tests/run-tests.py --exploration_strategy=exhaustive query_test/test_scanners_fuzz.py::TestScannersFuzzing::test_fuzz_decimal_tbl