[IMPALA-2480] test_spilling.py: Memory limit exceeded: Not enough memory to switch to IO-sized buffers for a partition of join 3.

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 2.3.0
• Fix Version/s: Impala 2.3.0
• Labels: broken-build
• Target Version: Impala 2.3.0
• Assignee: Daniel Hecht
• Reporter: Daniel Hecht
• Created: 05/Oct/15 19:14
• Updated: 12/Oct/15 00:29
• Resolved: 12/Oct/15 00:29

Description

test_spilling.py is still failing sometimes even though we reverted back to:

set max_block_mgr_memory=100m;

ippokratis, any advice on what we should do here? Do you think this might be a real regression?

http://sandbox.jenkins.cloudera.com/view/Impala/view/Builds%20-%202.3.0%20Release/job/impala-cdh5.5.x-exhaustive/29/testReport/junit/Impala.tests.custom_cluster.test_spillling/TestSpilling/test_spilling_exec_option____disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0___batch_size___0___num_nodes___0____table_format__parquet_none_/

-- executing against localhost:21000

set max_block_mgr_memory=100m;

-- executing against localhost:21000

select count(l1.l_tax)
from
lineitem l1,
lineitem l2,
lineitem l3
where
l1.l_tax < 0.01 and
l2.l_tax < 0.04 and
l1.l_orderkey = l2.l_orderkey and
l1.l_orderkey = l3.l_orderkey and
l1.l_comment = l3.l_comment and
l1.l_shipdate = l3.l_shipdate;

>           raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
E           ImpalaBeeswaxException: ImpalaBeeswaxException:
E            Query aborted:
E           Memory limit exceeded
E           Not enough memory to switch to IO-sized buffers for a partition of join 3.
E           
E           
E           
E           Memory Limit Exceeded
E           Query(94e5e5b0feccda8:24ef072cf87f45bb) Limit: Consumption=112.60 MB
E             Fragment 94e5e5b0feccda8:24ef072cf87f45bd: Consumption=103.97 MB
E               AGGREGATION_NODE (id=5): Consumption=4.00 KB
E               HASH_JOIN_NODE (id=4): Consumption=59.59 MB
E               HASH_JOIN_NODE (id=3): Consumption=44.35 MB
E               HDFS_SCAN_NODE (id=2): Consumption=0
E               EXCHANGE_NODE (id=6): Consumption=0
E               EXCHANGE_NODE (id=7): Consumption=0
E               DataStreamRecvr: Consumption=0
E               DataStreamSender: Consumption=16.00 KB
E             Block Manager: Limit=100.00 MB Consumption=99.92 MB
E             Fragment 94e5e5b0feccda8:24ef072cf87f45c3: Consumption=8.64 MB
E               HDFS_SCAN_NODE (id=0): Consumption=8.62 MB
E               DataStreamSender: Consumption=6.84 KB