Details
-
Bug
-
Status: Resolved
-
Blocker
-
Resolution: Fixed
-
Impala 2.7.0, Impala 2.8.0
Description
Steps to reproduce:
1. Compile the code from https://github.com/cloudera/impala-udf-samples to udf-samples.ll
2. Create a UDF using functions defined in the LLVM IR code.
3. Invoke the newly created UDF and Impala crashes with the following stack trace.
(gdb) bt
#0 0x00007f36d166ccc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007f36d16700d8 in __GI_abort () at abort.c:89
#2 0x0000000002821c04 in google::DumpStackTraceAndExit() ()
#3 0x000000000281b06d in google::LogMessage::Fail() ()
#4 0x000000000281d996 in google::LogMessage::SendToLog() ()
#5 0x000000000281ab8d in google::LogMessage::Flush() ()
#6 0x000000000281e43e in google::LogMessageFatal::~LogMessageFatal() ()
#7 0x0000000001627a46 in impala::LlvmCodeGen::LinkModule (this=0x883ff80, file=...) at /kwho-desktop-1/trees/incubator-impala/be/src/codegen/llvm-codegen.cc:368
#8 0x00000000018cb92a in impala::ScalarFnCall::GetCodegendComputeFn (this=0x8682600, codegen=0x883ff80, fn=0x7f369e211a78) at /kwho-desktop-1/trees/incubator-impala/be/src/exprs/scalar-fn-call.cc:314
#9 0x0000000001367f25 in impala::RuntimeState::CodegenScalarFns (this=0x7f369e211eb0) at /kwho-desktop-1/trees/incubator-impala/be/src/runtime/runtime-state.cc:197
#10 0x000000000146c3f4 in Java_org_apache_impala_service_FeSupport_NativeEvalConstExprs (env=0x73fb1e8, caller_class=0x7f369e212af0, thrift_expr_batch=0x7f369e212b08, thrift_query_ctx_bytes=0x7f369e212b00)
at /kwho-desktop-1/trees/incubator-impala/be/src/service/fe-support.cc:133
#11 0x00007f36cb49bc39 in ?? ()
#12 0x00000000073fb000 in ?? ()
#13 0x000000000000023f in ?? ()
#14 0x00007f369e212aa0 in ?? ()
#15 0x0000000606b3ab20 in ?? ()
#16 0x00007f369e212b08 in ?? ()
#17 0x0000000606b3c298 in ?? ()
#18 0x0000000000000000 in ?? ()
(gdb) f 7
#7 0x0000000001627a46 in impala::LlvmCodeGen::LinkModule (this=0x883ff80, file=...) at /kwho-desktop-1/trees/incubator-impala/be/src/codegen/llvm-codegen.cc:368
368 DCHECK(fn != NULL);
(gdb) l
363 // Parse materialized functions in the source module and materialize functions it
364 // references. Do it after linking so LLVM has "merged" functions defined in both
365 // modules.
366 for (const string& fn_name: materializable_fns) {
367 Function* fn = module_->getFunction(fn_name);
368 DCHECK(fn != NULL);
369 if (!fn->isMaterializable()) MaterializeCallees(fn);
370 }
371 return Status::OK();
372 }
(gdb) p fn_name
$1 = (const std::string &) @0x8d95930: {static npos = <optimized out>, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>},
_M_p = 0x84fbcd8 "_ZN10impala_udf9StringValC2EPhi"}}
The crash has to do with being unable to look up a function using its name in the linked module. The function's name was recorded by parsing the external LLVM module before it's linked to the 'main_module_' of Impala. Apparently, the name of the linked function may have been renamed during linking and it could no longer be found after linking.
cc'ing tarmstrong for his opinion.