[IMPALA-288] Excessive mem usage for certain queries which are very selective

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 0.7
• Fix Version/s: Impala 1.0
• Assignee: Nong Li
• Reporter: Nong Li
• Created: 18/Apr/13 22:21
• Updated: 23/Apr/13 21:17
• Resolved: 23/Apr/13 21:17

Description

We changed how row batches when row batches are used from the scanner threads to the scan node. They used to be pushed eagerly, causing row batches which have few tuples. Now they are pushed when the row batch is full, accumulating external (mem pool, io buffers) until they are full. For queries that return very few rows, the accumulated resources cause excessive resource usage.