[IMPALA-4541] test_codegen_mem_limit is flaky

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 2.8.0
• Fix Version/s: Impala 2.8.0
• Component/s: Backend
• Labels: broken-build
• Target Version: Impala 2.8.0
• Assignee: Tim Armstrong
• Reporter: Michael Ho
• Created: 25/Nov/16 07:35
• Updated: 28/Nov/16 18:22
• Resolved: 28/Nov/16 18:22

Description

The test seems to exceed memory limit at another location so it doesn't consistently fail due to exceeding memory limit during codegen.
The exhaustive RHEL7 build hit the following failure.

Error Message

query_test/test_query_mem_limit.py:123: in test_codegen_mem_limit     self.run_test_case('QueryTest/codegen-mem-limit', vector) common/impala_test_suite.py:324: in run_test_case     self.__verify_exceptions(test_section['CATCH'], str(e), use_db) common/impala_test_suite.py:215: in __verify_exceptions     (expected_str, actual_str) E   AssertionError: Unexpected exception string. Expected: Codegen failed to reserve E   Not found in actual: ImpalaBeeswaxException: Query aborted:Memory limit exceededFailed to allocate tuple bufferMemory Limit Exceeded by fragment: a44539d48545d15b:2  HDFS_SCAN_NODE (id=0) could not allocate 90.00 KB without exceeding limit.Query(a44539d48545d15b:ec0ea61000000000): Limit=100.00 KB Total=64.42 KB Peak=64.42 KB  Fragment a44539d48545d15b:2: Total=64.42 KB Peak=64.42 KB    HDFS_SCAN_NODE (id=0): Total=55.00 KB Peak=55.00 KB      Exprs: Total=8.00 KB Peak=8.00 KB    DataStreamSender (dst_id=1): Total=1.42 KB Peak=1.42 KB  Block Manager: Total=0 Peak=0
Stacktrace

query_test/test_query_mem_limit.py:123: in test_codegen_mem_limit
    self.run_test_case('QueryTest/codegen-mem-limit', vector)
common/impala_test_suite.py:324: in run_test_case
    self.__verify_exceptions(test_section['CATCH'], str(e), use_db)
common/impala_test_suite.py:215: in __verify_exceptions
    (expected_str, actual_str)
E   AssertionError: Unexpected exception string. Expected: Codegen failed to reserve
E   Not found in actual: ImpalaBeeswaxException: Query aborted:Memory limit exceededFailed to allocate tuple bufferMemory Limit Exceeded by fragment: a44539d48545d15b:2  HDFS_SCAN_NODE (id=0) could not allocate 90.00 KB without exceeding limit.Query(a44539d48545d15b:ec0ea61000000000): Limit=100.00 KB Total=64.42 KB Peak=64.42 KB  Fragment a44539d48545d15b:2: Total=64.42 KB Peak=64.42 KB    HDFS_SCAN_NODE (id=0): Total=55.00 KB Peak=55.00 KB      Exprs: Total=8.00 KB Peak=8.00 KB    DataStreamSender (dst_id=1): Total=1.42 KB Peak=1.42 KB  Block Manager: Total=0 Peak=0
Standard Error

-- connecting to: localhost:21000
-- executing against localhost:21000
use functional_parquet;

SET disable_codegen=True;
SET abort_on_error=1;
SET exec_single_node_rows_threshold=0;
SET batch_size=0;
SET num_nodes=0;
-- executing against localhost:21000
set mem_limit=100k;

-- executing against localhost:21000

select *
from alltypes
where substr(string_col, 1) = "";

-- executing against localhost:21000
SET MEM_LIMIT=0;;