[IMPALA-1910] Significant memory leak in HashJoin Node

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.1.1
• Fix Version/s: Impala 2.2
• Assignee: Daniel Hecht
• Reporter: Alan Choi
• Created: 23/Mar/15 18:59
• Updated: 20/Dec/15 00:05
• Resolved: 18/May/15 20:29

Description

Here's the leak:

Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
F0320 20:41:54.770300 28819 exec-node.cc:175] Check failed: mem_tracker()->consumption() == 0 (1539309568 vs. 0) Leaked memory.
Fragment af4cf08b2927df82:dc517e368ba78aa4: Consumption=1.43 GB
AGGREGATION_NODE (id=3): Consumption=0
HASH_JOIN_NODE (id=2): Consumption=1.43 GB
EXCHANGE_NODE (id=4): Consumption=0
EXCHANGE_NODE (id=5): Consumption=0
DataStreamSender: Consumption=16.00 KB

The leak is a lot more than what's reported in IMPALA-1867.

The query that caused the leak is in this form:

select * from small_tbl A, big_tbl B where A.id=B.id

The per query mem_limit is set to ~1.5gb.

We can use tpc-ds to try to repro it.

Issue Links

• relates to: IMPALA-1867 Crash: DCHECK: mem_tracker()->consumption() == 0 (8388608 vs. 0) Leaked memory.