[IMPALA-4787] Optimize APPX_MEDIAN() mem usage in case of many grouping keys

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.8.0
• Fix Version/s: Impala 2.9.0
• Component/s: Backend
• Labels: usability
• Target Version: Impala 2.9.0
• Assignee: Taras Bobrovytsky
• Reporter: Marcell Szabo
• Created: 19/Jan/17 14:22
• Updated: 25/Apr/17 12:07
• Resolved: 17/Mar/17 00:05

Description

APPX_MEDIAN uses a lot of memory per grouping key. It allocates space for 20,000 samples per grouping key to estimate the median. The current implementation targeted towards non-grouping aggregations or aggregations with relatively few distinct grouping keys.

Issue Links

• breaks: IMPALA-5088 heap-buffer-overflow in impala_udf::StringVal::CopyFrom
• relates to: IMPALA-5088 heap-buffer-overflow in impala_udf::StringVal::CopyFrom