[IMPALA-3053] limit 0; causes huge cluster_mem_estimate

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: Impala 2.3.0
• Fix Version/s: Impala 2.5.0
• Component/s: Backend
• Assignee: Unassigned
• Reporter: Marcell Szabo
• Created: 22/Feb/16 16:16
• Updated: 25/Feb/16 05:08
• Resolved: 24/Feb/16 23:19

Description

Some tools (e.g. Tableau) run their queries with "limit 0" first to conveniently get the list of column names.
In this specific case the Admission control calculates cluster_mem_estimate = (number of hosts)*(rm_default_memory=4G), which can quickly allocate all the memory in the pool, causing admission timeouts while nothing is really running.

The problem arises in admission-controller.cc GetPerHostMemoryEstimate().
request_.per_host_mem_req is 0 in this case, which is treated as an invalid value, thus has_estimate remains false, and the logic falls back to using FLAGS_rm_default_memory, which is by default 4G.