[IMPALA-3471] TopN should be able to spill

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: Impala 2.6.0
• Component/s: Backend
• Epic Link: Operator memory constraints
• Target Version: Product Backlog
• Assignee: Unassigned
• Reporter: Jim Apple
• Created: 03/May/16 17:59
• Updated: 13/Jun/20 01:08

Description

TopN nodes store OFFSET + LIMIT tuples in memory. (In fact, in a vector which will throw an exception if allocation fails.) It would be nice to check allocations before they fail and spill when there isn't enough memory.

Issue Links

• is related to: IMPALA-5004 Switch to sorting node for large TopN queries
• is related to: IMPALA-9853 Push rank() predicates into sort
• is related to: IMPALA-4995 crash when limit clause > MAX_INT