[IMPALA-6419] hdfs-parquet-scanner.cc:624] Check failed: 0 == context_->NumStreams() (0 vs. 11)

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 2.12.0
• Fix Version/s: Impala 2.12.0
• Component/s: Backend
• Labels: broken-build
• Target Version: Impala 2.12.0
• Epic Color: ghx-label-8
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 18/Jan/18 17:41
• Updated: 18/Jan/18 22:56
• Resolved: 18/Jan/18 22:56

Description

Hit this during GVO

https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/975/artifact/Impala/logs_static/logs/ee_tests/impalad.ip-172-31-46-11.ubuntu.log.ERROR.20180118-025218.43759

F0118 03:59:51.644770 21603 hdfs-parquet-scanner.cc:624] Check failed: 0 == context_->NumStreams() (0 vs. 11) 
*** Check failure stack trace: ***
    @          0x3c0611d  google::LogMessage::Fail()
    @          0x3c079c2  google::LogMessage::SendToLog()
    @          0x3c05af7  google::LogMessage::Flush()
    @          0x3c090be  google::LogMessageFatal::~LogMessageFatal()
    @          0x1bf5f26  impala::HdfsParquetScanner::NextRowGroup()
    @          0x1bf4c52  impala::HdfsParquetScanner::GetNextInternal()
    @          0x1bf3264  impala::HdfsParquetScanner::ProcessSplit()
    @          0x1b7cf42  impala::HdfsScanNode::ProcessSplit()
    @          0x1b7c36d  impala::HdfsScanNode::ScannerThread()
    @          0x1b7b7f0  _ZZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS_17ThreadResourceMgr12ResourcePoolEENKUlvE_clEv
    @          0x1b7d793  _ZN5boost6detail8function26void_function_obj_invoker0IZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS3_17ThreadResourceMgr12ResourcePoolEEUlvE_vE6invokeERNS1_15function_bufferE
    @          0x17d669a  boost::function0<>::operator()()
    @          0x1ad2993  impala::Thread::SuperviseThread()
    @          0x1adb6a8  boost::_bi::list4<>::operator()<>()
    @          0x1adb5eb  boost::_bi::bind_t<>::operator()()
    @          0x1adb5ae  boost::detail::thread_data<>::run()
    @          0x2d8faea  thread_proxy
    @     0x7f64f6b1f6ba  start_thread
    @     0x7f64f685541d  clone

Repro:

$ SCANNER_FUZZ_SEED=1516247966 impala-py.test tests/query_test/test_scanners_fuzz.py -n 4 --verbose -k parquet

Issue Links

• is broken by: IMPALA-6383 Memory from previous row groups can accumulate in Parquet scanner