[IMPALA-561] Executing some queries with NUM_NODES=1 results in Failed DCHECK in thread-resource-mgr.cc:91] Check failed: thread_available_fn_ == __null || fn == __null

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 1.2
• Fix Version/s: Impala 2.6.0
• Component/s: Backend
• Labels: resource-management
• Target Version: Product Backlog
• Assignee: Michael Ho
• Reporter: Lenni Kuff
• Created: 26/Aug/13 15:06
• Updated: 22/Aug/23 18:33
• Resolved: 10/Mar/16 23:57

Description

Looks like they are both set:

F0826 02:00:44.964058 19004 thread-resource-mgr.cc:91] Check failed: thread_available_fn_ == __null || fn == __null 

(gdb) print thread_available_fn_
$1 = {<boost::function1<void, impala::ThreadResourceMgr::ResourcePool*>> = {<boost::function_base> = {
      vtable = 0x24728d1, functor = {obj_ptr = 0x1213544, type = {type = 0x1213544, const_qualified = false, 
          volatile_qualified = false}, 
        func_ptr = 0x1213544 <impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourceMgr::ResourcePool*)>, bound_memfunc_ptr = {memfunc_ptr = (void (boost::detail::function::X::*)(boost::detail::function::X *, 
    int)) 0x1213544 <impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourceMgr::ResourcePool*)>, 
          obj_ptr = 0x62bd700}, obj_ref = {obj_ptr = 0x1213544, is_const_qualified = false, 
          is_volatile_qualified = false}, 
        data = 68 'D'}}, <std::unary_function<impala::ThreadResourceMgr::ResourcePool*, void>> = {<No data fields>}, static args = <optimized out>, static arity = <optimized out>}, <No data fields>}


(gdb) print fn
$2 = {<boost::function1<void, impala::ThreadResourceMgr::ResourcePool*>> = {<boost::function_base> = {
      vtable = 0x24728d1, functor = {obj_ptr = 0x1213544, type = {type = 0x1213544, const_qualified = false, 
          volatile_qualified = false}, 
        func_ptr = 0x1213544 <impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourceMgr::ResourcePool*)>, bound_memfunc_ptr = {memfunc_ptr = (void (boost::detail::function::X::*)(boost::detail::function::X *, 
    int)) 0x1213544 <impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourceMgr::ResourcePool*)>, 
          obj_ptr = 0x62ba000}, obj_ref = {obj_ptr = 0x1213544, is_const_qualified = false, 
          is_volatile_qualified = false}, 
        data = 68 'D'}}, <std::unary_function<impala::ThreadResourceMgr::ResourcePool*, void>> = {<No data fields>}, static args = <optimized out>, static arity = <optimized out>}, <No data fields>}
(gdb) 

#7  0x0000000000f9e110 in impala::ThreadResourceMgr::ResourcePool::SetThreadAvailableCb (this=0x5dc24e0, 
    fn=...)
    at /data/1/jenkins/workspace/impala-master-64bit-code-coverage/repos/Impala/be/src/runtime/thread-resource-mgr.cc:91
#8  0x0000000001210449 in impala::HdfsScanNode::Prepare (this=0x62ba000, state=0x6e9e480)
    at /data/1/jenkins/workspace/impala-master-64bit-code-coverage/repos/Impala/be/src/exec/hdfs-scan-node.cc:461
#9  0x00000000011dd57c in impala::ExecNode::Prepare (this=0x7e59b80, state=0x6e9e480)
    at /data/1/jenkins/workspace/impala-master-64bit-code-coverage/repos/Impala/be/src/exec/exec-node.cc:85
#10 0x00000000011eb439 in impala::HashJoinNode::Prepare (this=0x7e59b80, state=0x6e9e480)
    at /data/1/jenkins/workspace/impala-master-64bit-code-coverage/repos/Impala/be/src/exec/hash-join-node.cc:80
#11 0x0000000000f810df in impala::PlanFragmentExecutor::Prepare (this=0x2308b340, request=...)
    at /data/1/jenkins/workspace/impala-master-64bit-code-coverage/repos/Impala/be/src/runtime/plan-fragment-executor.cc:150
#12 0x0000000000ec659b in impala::Coordinator::Exec (this=0x5e90000, query_id=..., request=..., 
    query_options=..., output_exprs=0x7cf8800)
    at /data/1/jenkins/workspace/impala-master-64bit-code-coverage/repos/Impala/be/src/runtime/coordinator.cc:329
#13 0x0000000000c8fa82 in impala::ImpalaServer::QueryExecState::ExecQueryOrDmlRequest (this=0x7cf8000, 
    query_exec_request=...)
    at /data/1/jenkins/workspace/impala-master-64bit-code-coverage/repos/Impala/be/src/service/query-exec-state.cc:182
#14 0x0000000000c8e878 in impala::ImpalaServer::QueryExecState::Exec (this=0x7cf8000, 
    exec_request=0x7f140ef01c40)
    at /data/1/jenkins/workspace/impala-master-64bit-code-coverage/repos/Impala/be/src/service/query-exec-state.cc:123
#15 0x0000000000b92887 in impala::ImpalaServer::ExecuteInternal (this=0x53e3180, request=..., 
    session_state=..., query_session_state=..., registered_exec_state=0x7f140ef02537, 
    exec_state=0x7f140ef026b0)
    at /data/1/jenkins/workspace/impala-master-64bit-code-coverage/repos/Impala/be/src/service/impala-server.cc:949
#16 0x0000000000b91b4a in impala::ImpalaServer::Execute (this=0x53e3180, request=..., session_state=..., 
    query_session_state=..., exec_state=0x7f140ef026b0)
    at /data/1/jenkins/workspace/impala-master-64bit-code-coverage/repos/Impala/be/src/service/impala-server.cc:893
#17 0x0000000000c7a781 in impala::ImpalaServer::query (this=0x53e3180, query_handle=..., query=...)
    at /data/1/jenkins/workspace/impala-master-64bit-code-coverage/repos/Impala/be/src/service/impala-beeswax-server.cc:153
#18 0x000000000109b663 in beeswax::BeeswaxServiceProcessor::process_query (this=0x3ae1860, seqid=0, 
    iprot=0x2741c0c0, oprot=0x719e140, callContext=0x719f200)
    at /data/1/jenkins/workspace/impala-master-64bit-code-coverage/repos/Impala/be/generated-sources/gen-cpp/BeeswaxService.cpp:2979
#19 0x000000000109b235 in beeswax::BeeswaxServiceProcessor::dispatchCall (this=0x3ae1860, iprot=0x2741c0c0, 

logs and coredump at a2123.halxg.cloudera.com:/data/1/jenkins/workspaces/impala-561

Issue Links

• causes: IMPALA-12394 Restore testing single-node planning for most tests
• is duplicated by: IMPALA-2032 Impala crashes on join queries when num_nodes = 1
• is duplicated by: IMPALA-3064 thread-resource-mgr.cc:92] Check failed: thread_available_fn_ == __null || fn == __null