Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1894

Old (non-partitioned) hash join and aggregation do not work

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Impala 2.2
    • Fix Version/s: Impala 2.2
    • Component/s: None
    • Labels:
      None
    • Environment:
      git hash: a1171a3aff1d9eb2d4078521d10b6df0675d71a0

      Description

      I tried running tests with --enable_partitioned_hash_join=false, but got a crash. Looks like tcmalloc was corrupted. This is easy to reproduce by just running the end-to-end tests with that flag:

      #7  0x0000000001fa5828 in tcmalloc::CentralFreeList::RemoveRange (this=0x2856a60, start=0x7f21f8041e48, end=0x7f21f8041e40, N=32) at src/central_freelist.cc:269
      #8  0x0000000001fabd84 in tcmalloc::ThreadCache::FetchFromCentralCache (this=0xbbfc780, cl=<value optimized out>, byte_size=112) at src/thread_cache.cc:162
      #9  0x0000000001fbfec8 in Allocate (size=112) at src/thread_cache.h:341
      #10 do_malloc (size=112) at src/tcmalloc.cc:1068
      #11 cpp_alloc (size=112) at src/tcmalloc.cc:1354
      #12 tc_new (size=112) at src/tcmalloc.cc:1530
      #13 0x0000000001e9cb23 in llvm::User::operator new(unsigned long, unsigned int) ()
      #14 0x0000000000eb632a in llvm::UnaryInstruction::operator new (s=88) at /opt/toolchain/llvm-3.3/include/llvm/IR/InstrTypes.h:105
      #15 0x0000000001e791af in llvm::CastInst::Create(llvm::Instruction::CastOps, llvm::Value*, llvm::Type*, llvm::Twine const&, llvm::Instruction*) ()
      #16 0x00000000016861cb in llvm::BitcodeReader::ParseFunctionBody(llvm::Function*) ()
      #17 0x000000000168b9d6 in llvm::BitcodeReader::Materialize(llvm::GlobalValue*, std::basic_string<char, std::char_traits<char>, std::allocator<char> >*) ()
      #18 0x000000000168b70d in llvm::BitcodeReader::MaterializeModule(llvm::Module*, std::basic_string<char, std::char_traits<char>, std::allocator<char> >*) ()
      #19 0x0000000001e83599 in llvm::Module::MaterializeAllPermanently(std::basic_string<char, std::char_traits<char>, std::allocator<char> >*) ()
      #20 0x000000000168c871 in llvm::ParseBitcodeFile(llvm::MemoryBuffer*, llvm::LLVMContext&, std::basic_string<char, std::char_traits<char>, std::allocator<char> >*) ()
      #21 0x0000000000eaaf6a in impala::LlvmCodeGen::LoadModule (codegen=0x1356cc00, file="/home/dhecht/src/Impala/llvm-ir/impala-sse.ll", module=0x7f21f8042c48) at /home/dhecht/src/Impala/be/src/codegen/llvm-codegen.cc:150
      #22 0x0000000000eaabb9 in impala::LlvmCodeGen::LoadFromFile (pool=0x14b68e80, file="/home/dhecht/src/Impala/llvm-ir/impala-sse.ll", id="3d40cf943ea9140c:fcfb765bb88886b9", codegen=0x34911790)
          at /home/dhecht/src/Impala/be/src/codegen/llvm-codegen.cc:126
      #23 0x0000000000eab60c in impala::LlvmCodeGen::LoadImpalaIR (pool=0x14b68e80, id="3d40cf943ea9140c:fcfb765bb88886b9", codegen_ret=0x34911790) at /home/dhecht/src/Impala/be/src/codegen/llvm-codegen.cc:189
      #24 0x0000000000f7cbea in impala::RuntimeState::CreateCodegen (this=0x34911500) at /home/dhecht/src/Impala/be/src/runtime/runtime-state.cc:193
      #25 0x0000000000f7e0df in impala::RuntimeState::GetCodegen (this=0x34911500, codegen=0x7f21f8043148, initialize=true) at /home/dhecht/src/Impala/be/src/runtime/runtime-state.cc:316
      #26 0x0000000001584cf2 in impala::PartitionedAggregationNode::Prepare (this=0x23f76900, state=0x34911500) at /home/dhecht/src/Impala/be/src/exec/partitioned-aggregation-node.cc:102
      #27 0x00000000014d4f6a in impala::ExecNode::Prepare (this=0x8acf600, state=0x34911500) at /home/dhecht/src/Impala/be/src/exec/exec-node.cc:149
      #28 0x00000000015b78af in impala::SortNode::Prepare (this=0x8acf600, state=0x34911500) at /home/dhecht/src/Impala/be/src/exec/sort-node.cc:44
      #29 0x000000000149ea04 in impala::PlanFragmentExecutor::Prepare (this=0x7c5af10, request=...) at /home/dhecht/src/Impala/be/src/runtime/plan-fragment-executor.cc:233
      #30 0x00000000010e35c4 in impala::FragmentMgr::FragmentExecState::Prepare (this=0x7c5ad00, exec_params=...) at /home/dhecht/src/Impala/be/src/service/fragment-exec-state.cc:43
      #31 0x00000000010d9fad in impala::FragmentMgr::ExecPlanFragment (this=0x9c31140, exec_params=...) at /home/dhecht/src/Impala/be/src/service/fragment-mgr.cc:51
      #32 0x000000000101ac23 in impala::ImpalaInternalService::ExecPlanFragment (this=0x8f5ac30, return_val=..., params=...) at /home/dhecht/src/Impala/be/src/service/impala-internal-service.h:37
      #33 0x000000000130ac87 in impala::ImpalaInternalServiceProcessor::process_ExecPlanFragment (this=0x9c310e0, seqid=0, iprot=0x72de740, oprot=0x72de900, callContext=0x139e7380)
          at /home/dhecht/src/Impala/be/generated-sources/gen-cpp/ImpalaInternalService.cpp:949
      #34 0x000000000130aa2c in impala::ImpalaInternalServiceProcessor::dispatchCall (this=0x9c310e0, iprot=0x72de740, oprot=0x72de900, fname="ExecPlanFragment", seqid=0, callContext=0x139e7380)
          at /home/dhecht/src/Impala/be/generated-sources/gen-cpp/ImpalaInternalService.cpp:922
      #35 0x0000000001007bde in apache::thrift::TDispatchProcessor::process (this=0x9c310e0, in=..., out=..., connectionContext=0x139e7380)
          at /home/dhecht/src/Impala/thirdparty/thrift-0.9.0/build/include/thrift/TDispatchProcessor.h:121
      #36 0x0000000001f1a259 in apache::thrift::server::TThreadedServer::Task::run (this=0x9459f80) at src/thrift/server/TThreadedServer.cpp:70
      #37 0x0000000000f42c5b in impala::ThriftThread::RunRunnable (this=0x139e7440, runnable=..., promise=0x7f2277df0480) at /home/dhecht/src/Impala/be/src/rpc/thrift-thread.cc:61
      #38 0x0000000000f4440b in boost::_mfi::mf2<void, impala::ThriftThread, boost::shared_ptr<apache::thrift::concurrency::Runnable>, impala::Promise<unsigned long>*>::operator() (this=0x879ba10, p=0x139e7440, a1=...,
          a2=0x7f2277df0480) at /usr/include/boost/bind/mem_fn_template.hpp:280
      #39 0x0000000000f44264 in boost::_bi::list3<boost::_bi::value<impala::ThriftThread*>, boost::_bi::value<boost::shared_ptr<a
      

        Attachments

          Activity

            People

            • Assignee:
              mjacobs Matthew Jacobs
              Reporter:
              dhecht Dan Hecht
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: