[IMPALA-1346] Sorter fails with out-of-memory in queries with multiple spilling operators

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.0, Impala 2.1, Impala 2.2, Impala 2.3.0
• Fix Version/s: Impala 2.7.0, Impala 2.6.2
• Component/s: Backend
• Labels: resource-management
• Target Version: Product Backlog, Impala 2.6.2
• Assignee: Tim Armstrong
• Reporter: Nong Li
• Created: 03/Oct/14 05:02
• Updated: 01/May/18 21:13
• Resolved: 06/Jun/16 00:12

Description

An example of this is a join that is spilling followed by a sort (or aggregate) that also spills.

These queries use memory very ineffective causing many of them to fail. The blocking
operator starts accumulating memory and over time, the streaming operator will have none left (it is streaming so therefore gives up memory as it processes). As streaming operator eventually has too little memory to repartition and fails.

We need coordination within an query.

Workaround
Increase mem_limit for query or reduce query size (by processing only a subset of data at a time).

Issue Links

• is related to: IMPALA-3700 Introduce upper limit to hash table size or HashBuckets for Streaming agg
• relates to: IMPALA-3670 Block is not deleted by sorter in some cases when spilling is disabled