Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4595

Linking IR UDF module to main module crashes Impala

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • Impala 2.7.0, Impala 2.8.0
    • Impala 2.8.0
    • Backend

    Description

      Steps to reproduce:

      1. Compile the code from https://github.com/cloudera/impala-udf-samples to udf-samples.ll
      2. Create a UDF using functions defined in the LLVM IR code.
      3. Invoke the newly created UDF and Impala crashes with the following stack trace.

      (gdb) bt
      #0  0x00007f36d166ccc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
      #1  0x00007f36d16700d8 in __GI_abort () at abort.c:89
      #2  0x0000000002821c04 in google::DumpStackTraceAndExit() ()
      #3  0x000000000281b06d in google::LogMessage::Fail() ()
      #4  0x000000000281d996 in google::LogMessage::SendToLog() ()
      #5  0x000000000281ab8d in google::LogMessage::Flush() ()
      #6  0x000000000281e43e in google::LogMessageFatal::~LogMessageFatal() ()
      #7  0x0000000001627a46 in impala::LlvmCodeGen::LinkModule (this=0x883ff80, file=...) at /kwho-desktop-1/trees/incubator-impala/be/src/codegen/llvm-codegen.cc:368
      #8  0x00000000018cb92a in impala::ScalarFnCall::GetCodegendComputeFn (this=0x8682600, codegen=0x883ff80, fn=0x7f369e211a78) at /kwho-desktop-1/trees/incubator-impala/be/src/exprs/scalar-fn-call.cc:314
      #9  0x0000000001367f25 in impala::RuntimeState::CodegenScalarFns (this=0x7f369e211eb0) at /kwho-desktop-1/trees/incubator-impala/be/src/runtime/runtime-state.cc:197
      #10 0x000000000146c3f4 in Java_org_apache_impala_service_FeSupport_NativeEvalConstExprs (env=0x73fb1e8, caller_class=0x7f369e212af0, thrift_expr_batch=0x7f369e212b08, thrift_query_ctx_bytes=0x7f369e212b00)
          at /kwho-desktop-1/trees/incubator-impala/be/src/service/fe-support.cc:133
      #11 0x00007f36cb49bc39 in ?? ()
      #12 0x00000000073fb000 in ?? ()
      #13 0x000000000000023f in ?? ()
      #14 0x00007f369e212aa0 in ?? ()
      #15 0x0000000606b3ab20 in ?? ()
      #16 0x00007f369e212b08 in ?? ()
      #17 0x0000000606b3c298 in ?? ()
      #18 0x0000000000000000 in ?? ()
      (gdb) f 7
      #7  0x0000000001627a46 in impala::LlvmCodeGen::LinkModule (this=0x883ff80, file=...) at /kwho-desktop-1/trees/incubator-impala/be/src/codegen/llvm-codegen.cc:368
      368         DCHECK(fn != NULL);
      (gdb) l
      363       // Parse materialized functions in the source module and materialize functions it
      364       // references. Do it after linking so LLVM has "merged" functions defined in both
      365       // modules.
      366       for (const string& fn_name: materializable_fns) {
      367         Function* fn = module_->getFunction(fn_name);
      368         DCHECK(fn != NULL);
      369         if (!fn->isMaterializable()) MaterializeCallees(fn);
      370       }
      371       return Status::OK();
      372     }
      
      (gdb) p fn_name
      $1 = (const std::string &) @0x8d95930: {static npos = <optimized out>, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>},
          _M_p = 0x84fbcd8 "_ZN10impala_udf9StringValC2EPhi"}}
      

      The crash has to do with being unable to look up a function using its name in the linked module. The function's name was recorded by parsing the external LLVM module before it's linked to the 'main_module_' of Impala. Apparently, the name of the linked function may have been renamed during linking and it could no longer be found after linking.

      cc'ing tarmstrong for his opinion.

      Attachments

        Activity

          People

            kwho Michael Ho
            kwho Michael Ho
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: