[IMPALA-3332] Sorter does not do query maintenance correctly

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 2.6.0
• Fix Version/s: Impala 2.6.0
• Component/s: Backend
• Labels: resource-management usability
• Target Version: Impala 2.6.0
• Assignee: Michael Ho
• Reporter: Tim Armstrong
• Created: 12/Apr/16 21:45
• Updated: 19/May/16 17:57
• Resolved: 19/May/16 17:57

Description

I ran this query on tpch_parquet:

select * from (select l_orderkey, l_returnflag, l_linestatus, '' from lineitem union all select l_orderkey, l_returnflag, l_linestatus, '' from lineitem union all select l_orderkey, l_returnflag, l_linestatus, '' from lineitem union all select l_orderkey, l_returnflag, l_linestatus, '' from lineitem) x order by l_orderkey

Even after it was running for 10+ seconds I didn't see any updates in the execution summary.

I ran a slightly different query which had runaway memory usage, I presume because it didn't free local allocations

select * from (select *, first_value(col) over (order by sort_col) fv from (select concat(l_linestatus, repeat('a', 2047)) sort_col, substr(l_returnflag, 1, 0) col from lineitem) q) q2 where fv != ''

All of the allocations were this call stack:

Sorter::TupleSorter::Partition
codegened function
impala_udf::StringVal
FunctionContextImpl::AllocateLocal