[IMPALA-4556] Crash in HdfsParquetScanner destructor with mt_dop > 0

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Duplicate
• Affects Version/s: Impala 2.8.0
• Component/s: Backend
• Labels: crash
• Target Version: Impala 2.8.0
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 29/Nov/16 22:49
• Updated: 29/Nov/16 22:56
• Resolved: 29/Nov/16 22:56

Description

I was investigating IMPALA-4554 and attached a debugger to the running process. This caused the client to timeout and the query to get cancelled, at which point it hit this DCHECK:

Running on machine: tarmstrong-box
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E1129 14:37:27.831372 27312 logging.cc:121] stderr will be logged to this file.
F1129 14:43:36.144094 27659 mem-pool.cc:69] Check failed: chunks_.empty() Must call FreeAll() or AcquireData() for this pool
*** Check failure stack trace: ***
    @          0x2ae2e46  google::DumpStackTraceAndExit()
    @          0x2adc2bd  google::LogMessage::Fail()
    @          0x2adebe6  google::LogMessage::SendToLog()
    @          0x2adbddd  google::LogMessage::Flush()
    @          0x2adf68e  google::LogMessageFatal::~LogMessageFatal()
    @          0x13d830d  impala::MemPool::~MemPool()
    @          0x15e7806  boost::checked_delete<>()
    @          0x17ef1a3  impala::HdfsParquetScanner::~HdfsParquetScanner()
    @          0x17ef22e  impala::HdfsParquetScanner::~HdfsParquetScanner()
    @          0x178bb63  boost::scoped_ptr<>::reset()
    @          0x17a73b9  impala::HdfsScanNodeMt::Close()
    @          0x173b447  impala::ExecNode::Close()
    @          0x1c84766  impala::PlanFragmentExecutor::Close()
    @          0x158b6ff  impala::FragmentMgr::FragmentExecState::Exec()
    @          0x157e376  impala::FragmentMgr::FragmentThread()
    @          0x15839fe  boost::_mfi::mf1<>::operator()()
    @          0x1583858  boost::_bi::list2<>::operator()<>()
    @          0x1583708  boost::_bi::bind_t<>::operator()()
    @          0x12e91f3  boost::function0<>::operator()()
    @          0x16b0846  impala::Thread::SuperviseThread()
    @          0x16b962b  boost::_bi::list4<>::operator()<>()
    @          0x16b94b8  boost::_bi::bind_t<>::operator()()
    @          0x1cf97ca  thread_proxy
    @     0x7f04552c170a  start_thread
    @     0x7f0454de182d  (unknown)
Picked up JAVA_TOOL_OPTIONS: -agentlib:jdwp=transport=dt_socket,address=30000,server=y,suspend=n 
Wrote minidump to /home/tarmstrong/Impala/incubator-impala/logs/cluster/minidumps/impalad/1c9f5990-c9a3-f768-19a81f12-79c73058.dmp

Issue Links

• duplicates: IMPALA-4458 DCHECK when running test_cancellation.py with MT_DOP > 0