[IMPALA-11000] DHECK hit in FillScratchMicroBatches

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.1.0
• Component/s: Backend
• Epic Color: ghx-label-10
• Assignee: Amogh Margoor
• Reporter: Csaba Ringhofer
• Created: 02/Nov/21 13:56
• Updated: 03/Dec/21 05:36
• Resolved: 03/Nov/21 16:27

Description

Happened during exhaustive tests.

Log:

F1101 16:05:08.671600  1042 hdfs-parquet-scanner.cc:2397] 66490317b236519e:1cc2e03a00000000] Check failed: scratch_batch_->AtEnd() 

Query:

I1101 16:05:02.796250   367 Frontend.java:1637] 66490317b236519e:1cc2e03a00000000] Analyzing query: create table ctas_cancel stored as parquetfile as SELECT STRAIGHT_JOIN *
           FROM lineitem
                  JOIN /*+broadcast*/ orders ON o_orderkey = l_orderkey
                  JOIN supplier ON s_suppkey = l_suppkey
           WHERE o_orderstatus = 'F'
           ORDER BY l_orderkey
           LIMIT 10000 db: tpch_parquet

The query should come from around here:
https://github.com/apache/impala/blob/master/tests/query_test/test_cancellation.py#L149

Callstack:

 
3 impalad!impala::HdfsParquetScanner::FillScratchMicroBatches(std::vector<impala::ParquetColumnReader*, std::allocator<impala::ParquetColumnReader*> > const&, impala::RowBatch*, bool*, impala::ScratchMicroBatch const*, int, int, int*) [hdfs-parquet-scanner.cc : 2397 + 0xf]
 4  impalad!impala::Status impala::HdfsParquetScanner::AssembleRows<false>(impala::RowBatch*, bool*) [hdfs-parquet-scanner.cc : 2287 + 0x6d]
 5  impalad!impala::HdfsParquetScanner::GetNextInternal(impala::RowBatch*) [hdfs-parquet-scanner.cc : 539 + 0x2b]
 6  impalad!impala::HdfsParquetScanner::ProcessSplit() [hdfs-parquet-scanner.cc : 427 + 0x39]
 7  impalad!impala::HdfsScanNode::ProcessSplit(std::vector<impala::FilterContext, std::allocator<impala::FilterContext> > const&, impala::MemPool*, impala::io::ScanRange*, long*) [hdfs-scan-node.cc : 500 + 0x28]

Issue Links

• is broken by: IMPALA-9873 Skip decoding of non-materialised columns in Parquet