Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4595

Linking IR UDF module to main module crashes Impala

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Impala 2.7.0, Impala 2.8.0
    • Fix Version/s: Impala 2.8.0
    • Component/s: Backend
    • Labels:

      Description

      Steps to reproduce:

      1. Compile the code from https://github.com/cloudera/impala-udf-samples to udf-samples.ll
      2. Create a UDF using functions defined in the LLVM IR code.
      3. Invoke the newly created UDF and Impala crashes with the following stack trace.

      (gdb) bt
      #0  0x00007f36d166ccc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
      #1  0x00007f36d16700d8 in __GI_abort () at abort.c:89
      #2  0x0000000002821c04 in google::DumpStackTraceAndExit() ()
      #3  0x000000000281b06d in google::LogMessage::Fail() ()
      #4  0x000000000281d996 in google::LogMessage::SendToLog() ()
      #5  0x000000000281ab8d in google::LogMessage::Flush() ()
      #6  0x000000000281e43e in google::LogMessageFatal::~LogMessageFatal() ()
      #7  0x0000000001627a46 in impala::LlvmCodeGen::LinkModule (this=0x883ff80, file=...) at /kwho-desktop-1/trees/incubator-impala/be/src/codegen/llvm-codegen.cc:368
      #8  0x00000000018cb92a in impala::ScalarFnCall::GetCodegendComputeFn (this=0x8682600, codegen=0x883ff80, fn=0x7f369e211a78) at /kwho-desktop-1/trees/incubator-impala/be/src/exprs/scalar-fn-call.cc:314
      #9  0x0000000001367f25 in impala::RuntimeState::CodegenScalarFns (this=0x7f369e211eb0) at /kwho-desktop-1/trees/incubator-impala/be/src/runtime/runtime-state.cc:197
      #10 0x000000000146c3f4 in Java_org_apache_impala_service_FeSupport_NativeEvalConstExprs (env=0x73fb1e8, caller_class=0x7f369e212af0, thrift_expr_batch=0x7f369e212b08, thrift_query_ctx_bytes=0x7f369e212b00)
          at /kwho-desktop-1/trees/incubator-impala/be/src/service/fe-support.cc:133
      #11 0x00007f36cb49bc39 in ?? ()
      #12 0x00000000073fb000 in ?? ()
      #13 0x000000000000023f in ?? ()
      #14 0x00007f369e212aa0 in ?? ()
      #15 0x0000000606b3ab20 in ?? ()
      #16 0x00007f369e212b08 in ?? ()
      #17 0x0000000606b3c298 in ?? ()
      #18 0x0000000000000000 in ?? ()
      (gdb) f 7
      #7  0x0000000001627a46 in impala::LlvmCodeGen::LinkModule (this=0x883ff80, file=...) at /kwho-desktop-1/trees/incubator-impala/be/src/codegen/llvm-codegen.cc:368
      368         DCHECK(fn != NULL);
      (gdb) l
      363       // Parse materialized functions in the source module and materialize functions it
      364       // references. Do it after linking so LLVM has "merged" functions defined in both
      365       // modules.
      366       for (const string& fn_name: materializable_fns) {
      367         Function* fn = module_->getFunction(fn_name);
      368         DCHECK(fn != NULL);
      369         if (!fn->isMaterializable()) MaterializeCallees(fn);
      370       }
      371       return Status::OK();
      372     }
      
      (gdb) p fn_name
      $1 = (const std::string &) @0x8d95930: {static npos = <optimized out>, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>},
          _M_p = 0x84fbcd8 "_ZN10impala_udf9StringValC2EPhi"}}
      

      The crash has to do with being unable to look up a function using its name in the linked module. The function's name was recorded by parsing the external LLVM module before it's linked to the 'main_module_' of Impala. Apparently, the name of the linked function may have been renamed during linking and it could no longer be found after linking.

      cc'ing Tim Armstrong for his opinion.

        Attachments

          Activity

            People

            • Assignee:
              kwho Michael Ho
              Reporter:
              kwho Michael Ho
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: