[IMPALA-5911] Grouping aggregations with having conjuncts and Serialize()/Finalize() functions uses excessive expr memory

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: Impala 2.10.0
• Fix Version/s: Impala 2.11.0
• Component/s: Backend
• Target Version: Impala 2.11.0
• Epic Color: ghx-label-7
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 08/Sep/17 18:10
• Updated: 13/Oct/17 01:21
• Resolved: 13/Oct/17 01:21

Description

The loop in GetRowsFromPartition() invokes Finalize()/Serialize() on all of the rows in output_partition_, which may allocate local allocations in output_partition_->agg_fn_evals. These are not cleared out until the partition is destroyed.

If the having conjuncts are very select this can result in a lot of excess memory. E.g. the following query results in a lot of non-buffer-pool memory overhead:

select l_orderkey, min(l_comment) from tpch_parquet.lineitem group by l_orderkey having min(l_comment) like '%slow dolphins%' order by l_orderkey

Issue Links

• relates to: IMPALA-5844 Fix management of FunctionContext "local" allocations.