Details
-
Bug
-
Status: Open
-
Trivial
-
Resolution: Unresolved
-
Impala 2.11.0
-
None
-
None
-
ghx-label-8
Description
While investigating IMPALA-6764, I tripped on the following crash after passing --llvm_cpu_attr_whitelist='' startup option to impalad. We should probably guard against this empty argument string. This happened with LLVM 3.9. I haven't tried the experiment with the latest code, which has LLVM 5.0.1. It looks like a DCHECK/CHECK kind of crash in LLVM:
(gdb) bt
#0 0x00007f4821417428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1 0x00007f482141902a in __GI_abort () at abort.c:89
#2 0x00000000020ee124 in google::DumpStackTraceAndExit() ()
#3 0x00000000020e4b9d in google::LogMessage::Fail() ()
#4 0x00000000020e6442 in google::LogMessage::SendToLog() ()
#5 0x00000000020e4577 in google::LogMessage::Flush() ()
#6 0x00000000020e7b3e in google::LogMessageFatal::~LogMessageFatal() ()
#7 0x0000000000d74f70 in impala::LlvmCodegenHandleError (user_data=<optimized out>, reason=..., gen_crash_diag=<optimized out>)
at /home/zoram/Impala/be/src/codegen/llvm-codegen.cc:126
#8 0x0000000002023baa in llvm::report_fatal_error(llvm::Twine const&, bool) ()
#9 0x0000000002023d4e in llvm::report_fatal_error(std::string const&, bool) ()
#10 0x0000000001691c9a in llvm::SelectionDAGISel::CannotYetSelect(llvm::SDNode*) ()
#11 0x00000000016955bf in llvm::SelectionDAGISel::SelectCodeCommon(llvm::SDNode*, unsigned char const*, unsigned int) ()
#12 0x0000000001494f8f in (anonymous namespace)::X86DAGToDAGISel::Select(llvm::SDNode*) ()
#13 0x0000000001690649 in llvm::SelectionDAGISel::DoInstructionSelection() ()
#14 0x0000000001693301 in llvm::SelectionDAGISel::CodeGenAndEmitDAG() ()
#15 0x000000000169a674 in llvm::SelectionDAGISel::SelectAllBasicBlocks(llvm::Function const&) ()
#16 0x000000000169c4ff in llvm::SelectionDAGISel::runOnMachineFunction(llvm::MachineFunction&) ()
#17 0x0000000001498424 in (anonymous namespace)::X86DAGToDAGISel::runOnMachineFunction(llvm::MachineFunction&) ()
#18 0x00000000018167d9 in llvm::MachineFunctionPass::runOnFunction(llvm::Function&) ()
#19 0x0000000001fac06a in llvm::FPPassManager::runOnFunction(llvm::Function&) ()
#20 0x0000000001fac693 in llvm::legacy::PassManagerImpl::run(llvm::Module&) ()
#21 0x0000000001c9d3a9 in llvm::MCJIT::emitObject(llvm::Module*) ()
#22 0x0000000001c9d8b5 in llvm::MCJIT::generateCodeForModule(llvm::Module*) ()
#23 0x0000000001c9a180 in llvm::MCJIT::finalizeObject() ()
#24 0x0000000000d834c2 in impala::LlvmCodeGen::FinalizeModule (this=0xb73e580) at /home/zoram/Impala/be/src/codegen/llvm-codegen.cc:1120
#25 0x0000000000b8a7b8 in impala::FragmentInstanceState::Open (this=this@entry=0x7ccf4a0) at /home/zoram/Impala/be/src/runtime/fragment-instance-state.cc:254
#26 0x0000000000b8bcdf in impala::FragmentInstanceState::Exec (this=this@entry=0x7ccf4a0) at /home/zoram/Impala/be/src/runtime/fragment-instance-state.cc:81
#27 0x0000000000b7ad96 in impala::QueryState::ExecFInstance (this=0x67bcd00, fis=0x7ccf4a0) at /home/zoram/Impala/be/src/runtime/query-state.cc:386
#28 0x0000000000d3504f in boost::function0<void>::operator() (this=0x7f47bbae4d50)
at /workspace/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
#29 impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*) (name=..., category=..., functor=...,
thread_started=<optimized out>) at /home/zoram/Impala/be/src/util/thread.cc:354
#30 0x0000000000d35843 in boost::_bi::list4<boost::_bi::value<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<boost::function<void()> >, boost::_bi::value<impala::Promise<long int>*> >::operator()<void (*)(const std::basic_string<char>&, const std::basic_string<char>&, boost::function<void()>, impala::Promise<long int>*), boost::_bi::list0> (
f=@0x6b65db8: 0xd34dc0 <impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*)>,
a=<synthetic pointer>, this=0x6b65dc0) at /workspace/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:457
#31 boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> > >::operator()() (this=0x6b65db8)
at /workspace/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20
#32 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> > > >::run() (this=0x6b65c00) at /workspace/Impala/toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116
#33 0x00000000012cea8a in thread_proxy ()
#34 0x00007f48217b36ba in start_thread (arg=0x7f47bbae5700) at pthread_create.c:333
#35 0x00007f48214e941d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
We see the following in the Impalad log:
I0425 10:30:49.423234 6135 hdfs-scan-node.cc:161] Max row batch queue size for scan node '27' in fragment instance 'c547117b14900c82:133359b30000000a': 50
I0425 10:30:49.433096 6136 coordinator.cc:789] Coordinator waiting for backends to finish, 1 remaining
F0425 10:30:49.454378 6128 llvm-codegen.cc:126] LLVM hit fatal error: Cannot select: 0x758ab60: i64,glue = X86ISD::CMOV Constant:i64<0>, 0x758a070, Constant:i8<4>, 0x758a230
0x758a2a0: i64 = Constant<0>
0x758a070: i64,ch = CopyFromReg 0x73c1a70, Register:i64 %vreg3
0x758a380: i64 = Register %vreg3
0x758a5b0: i8 = Constant<4>
0x758a230: i32 = X86ISD::CMP 0x758a930, Constant:i64<0>
0x758a930: i64 = AssertZext 0x758a150, ValueType:ch:i1
0x758a150: i64,ch = CopyFromReg 0x73c1a70, Register:i64 %vreg2
0x758a4d0: i64 = Register %vreg2
0x758a2a0: i64 = Constant<0>
In function: impala::StringFunctions::LengthWrapper
The LLVM code is here: http://llvm.org/doxygen/SelectionDAGISel_8cpp_source.html