[IMPALA-3964] DCHECK hit for query involving count over nested collection

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 2.7.0
• Fix Version/s: Impala 2.7.0
• Component/s: Backend
• Labels: crash ramp-up
• Target Version: Impala 2.7.0
• Assignee: Chris Channing
• Reporter: Chris Channing
• Created: 09/Aug/16 13:38
• Updated: 04/Jan/17 23:58
• Resolved: 16/Aug/16 12:02

Description

Picked this up by accident via a typo in the subquery (should have been correlated). I've attached a sample Parquet file that I ran the query over.

select name, fr.cnt from foo f, (select count(*) cnt from foo.friends) fr;

Plan

                                                                
| F00:PLAN FRAGMENT [UNPARTITIONED]   
|   03:NESTED LOOP JOIN [CROSS JOIN]   
|   |  hosts=1 per-host-mem=unavailable 
|   |  tuple-ids=2,0 row-size=23B cardinality=unavailable
|   |                                      
|   |--00:SCAN HDFS [default.foo f]                                                  
|   |     partitions=1/1 files=1 size=9.49KB                                         
|   |     table stats: unavailable                                                   
|   |     column stats: unavailable                                                  
|   |     hosts=1 per-host-mem=unavailable                                           
|   |     tuple-ids=0 row-size=15B cardinality=unavailable                           
|   |                                                                                
|   02:AGGREGATE [FINALIZE]                                                          
|   |  output: count(*)                                                              
|   |  hosts=1 per-host-mem=unavailable                                              
|   |  tuple-ids=2 row-size=8B cardinality=1                                         
|   |                                                                                
|   01:SCAN HDFS [default.foo.friends]                                               
|      partitions=1/1 files=1 size=9.49KB                                            
|      table stats: unavailable                                                      
|      column stats: all                                                             
|      hosts=1 per-host-mem=unavailable                                              
|      tuple-ids=1 row-size=0B cardinality=unavailable                               

Trace
10 F0809 14:28:56.069478 12261 hdfs-parquet-scanner.cc:564] Check failed: scratch_batch_>mem_pool()>total_allocated_bytes() == 0 (24 vs. 0)
11 *** Check failure stack trace: ***
12 @ 0x281441d google::LogMessage::Fail()
13 @ 0x2816d46 google::LogMessage::SendToLog()
14 @ 0x2813f3d google::LogMessage::Flush()
15 @ 0x28177ee google::LogMessageFatal::~LogMessageFatal()
16 @ 0x16d89a5 impala::HdfsParquetScanner::TransferScratchTuples()
17 @ 0x16d75c0 impala::HdfsParquetScanner::AssembleRows()
18 @ 0x16d609c impala::HdfsParquetScanner::GetNextInternal()
19 @ 0x16d4f01 impala::HdfsParquetScanner::ProcessSplit()
20 @ 0x1690362 impala::HdfsScanNode::ProcessSplit()
21 @ 0x168f458 impala::HdfsScanNode::ScannerThread()
22 @ 0x16ab683 boost::_mfi::mf0<>::operator()()
23 @ 0x16aa806 boost::_bi::list1<>::operator()<>()
24 @ 0x16a92b5 boost::_bi::bind_t<>::operator()()
25 @ 0x16a70fe boost::detail::function::void_function_obj_invoker0<>::invoke()
26 @ 0x134f4fc boost::function0<>::operator()()
27 @ 0x15faafb impala::Thread::SuperviseThread()
28 @ 0x1601514 boost::_bi::list4<>::operator()<>()
29 @ 0x1601457 boost::_bi::bind_t<>::operator()()
30 @ 0x16013b2 boost::detail::thread_data<>::run()
31 @ 0x1a4622a thread_proxy
32 @ 0x2b43341a0184 start_thread
33 @ 0x2b4336f6637d (unknown)

Issue Links

• relates to: IMPALA-3952 Crash in HdfsParquetScanner::Close()