[IMPALA-2834] TestNestedTypes.test_tpch fails with memory limit exceeded - ASF JIRA

Details

Type: Bug
Status: Resolved
Priority: Minor
Resolution: Cannot Reproduce
Affects Version/s: Impala 2.5.0
Fix Version/s: None
Component/s: Backend
Labels:
- resource-management

Target Version:

Product Backlog

Description

http://sandbox.jenkins.cloudera.com/job/impala-master-cdh5-trunk/1631/testReport/junit/query_test.test_nested_types/TestNestedTypes/test_tpch_exec_option____disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0___batch_size___0___num_nodes___0____table_format__parquet_none_/

The scan node was using too much memory. I think this is probably just non-determinism in scan memory usage and the heuristics around it, i.e. ~~IMPALA-2831~~ , but we should keep an eye on this test to see if there is a possible actual regression, or if we need to fix the flaky test.

Error Message

query_test/test_nested_types.py:64: in test_tpch     self.run_test_case('QueryTest/nested-types-tpch', vector) common/impala_test_suite.py:216: in run_test_case     result = self.__execute_query(target_impalad_client, query, user=user) common/impala_test_suite.py:406: in __execute_query     return impalad_client.execute(query, user=user) common/impala_connection.py:158: in execute     return self.__beeswax_client.execute(sql_stmt, user=user) beeswax/impala_beeswax.py:162: in execute     handle = self.__execute_query(query_string.strip(), user=user) beeswax/impala_beeswax.py:328: in __execute_query     self.wait_for_completion(handle) beeswax/impala_beeswax.py:348: in wait_for_completion     raise ImpalaBeeswaxException("Query aborted:" + error_log, None) E   ImpalaBeeswaxException: ImpalaBeeswaxException: E    Query aborted: E   Memory limit exceeded E   Failed to allocate buffer for collection 'tpch_nested_parquet.customer.c_orders.item.o_lineitems'. E    E    E    E   Memory Limit Exceeded E   Query(6b4a1c66eac9b1e7:fc0c1c45d0a518a1) Limit: Limit=1.00 GB Consumption=1020.76 MB E     Fragment 6b4a1c66eac9b1e7:fc0c1c45d0a518a2: Consumption=8.00 KB E       EXCHANGE_NODE (id=10): Consumption=0 E       DataStreamRecvr: Consumption=0 E     Block Manager: Limit=819.20 MB Consumption=0 E     Fragment 6b4a1c66eac9b1e7:fc0c1c45d0a518a3: Consumption=1020.75 MB E       SORT_NODE (id=9): Consumption=28.00 KB E       SUBPLAN_NODE (id=1): Consumption=14.89 MB E       HDFS_SCAN_NODE (id=0): Consumption=1005.78 MB E       NESTED_LOOP_JOIN_NODE (id=8): Consumption=24.00 KB E       SUBPLAN_NODE (id=4): Consumption=8.00 KB E       UNNEST_NODE (id=3): Consumption=0 E       NESTED_LOOP_JOIN_NODE (id=7): Consumption=16.00 KB E       UNNEST_NODE (id=6): Consumption=0 E       SINGULAR_ROW_SRC_NODE (id=5): Consumption=0 E       SINGULAR_ROW_SRC_NODE (id=2): Consumption=0 E       DataStreamSender: Consumption=320.00 B

Stacktrace

query_test/test_nested_types.py:64: in test_tpch
    self.run_test_case('QueryTest/nested-types-tpch', vector)
common/impala_test_suite.py:216: in run_test_case
    result = self.__execute_query(target_impalad_client, query, user=user)
common/impala_test_suite.py:406: in __execute_query
    return impalad_client.execute(query, user=user)
common/impala_connection.py:158: in execute
    return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:162: in execute
    handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:328: in __execute_query
    self.wait_for_completion(handle)
beeswax/impala_beeswax.py:348: in wait_for_completion
    raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
E   ImpalaBeeswaxException: ImpalaBeeswaxException:
E    Query aborted:
E   Memory limit exceeded
E   Failed to allocate buffer for collection 'tpch_nested_parquet.customer.c_orders.item.o_lineitems'.
E   
E   
E   
E   Memory Limit Exceeded
E   Query(6b4a1c66eac9b1e7:fc0c1c45d0a518a1) Limit: Limit=1.00 GB Consumption=1020.76 MB
E     Fragment 6b4a1c66eac9b1e7:fc0c1c45d0a518a2: Consumption=8.00 KB
E       EXCHANGE_NODE (id=10): Consumption=0
E       DataStreamRecvr: Consumption=0
E     Block Manager: Limit=819.20 MB Consumption=0
E     Fragment 6b4a1c66eac9b1e7:fc0c1c45d0a518a3: Consumption=1020.75 MB
E       SORT_NODE (id=9): Consumption=28.00 KB
E       SUBPLAN_NODE (id=1): Consumption=14.89 MB
E       HDFS_SCAN_NODE (id=0): Consumption=1005.78 MB
E       NESTED_LOOP_JOIN_NODE (id=8): Consumption=24.00 KB
E       SUBPLAN_NODE (id=4): Consumption=8.00 KB
E       UNNEST_NODE (id=3): Consumption=0
E       NESTED_LOOP_JOIN_NODE (id=7): Consumption=16.00 KB
E       UNNEST_NODE (id=6): Consumption=0
E       SINGULAR_ROW_SRC_NODE (id=5): Consumption=0
E       SINGULAR_ROW_SRC_NODE (id=2): Consumption=0
E       DataStreamSender: Consumption=320.00 B

Standard Error

-- executing against localhost:21000
use functional_parquet;

SET disable_codegen=False;
SET abort_on_error=1;
SET exec_single_node_rows_threshold=0;
SET batch_size=0;
SET num_nodes=0;
-- executing against localhost:21000
use tpch_nested_parquet;

-- executing against localhost:21000
select count(*)
from tpch_nested_parquet.customer c, c.c_orders o, o.o_lineitems i
where c_custkey in (1, 2) and o_orderkey in (4808192, 1374019);

-- executing against localhost:21000
select count(*)
from tpch_nested_parquet.customer c, c.c_orders o, o.o_lineitems i;

-- executing against localhost:21000
select count(l_linenumber)
from tpch_nested_parquet.customer.c_orders.o_lineitems i;

-- executing against localhost:21000
select count(*)
from tpch_nested_parquet.customer.c_orders.o_lineitems i;

-- executing against localhost:21000
select count(l_linenumber) from tpch_parquet.lineitem;

-- executing against localhost:21000
select c_custkey, count(*)
from tpch_parquet.customer join tpch_parquet.orders on c_custkey = o_custkey
where c_custkey < 10
group by c_custkey
order by c_custkey;

-- executing against localhost:21000
select c_custkey from tpch_nested_parquet.customer c left anti join c.c_orders
where c_custkey < 10 order by c_custkey;

-- executing against localhost:21000
select c_custkey, c_name, o_orderkey, o_orderpriority
from tpch_nested_parquet.customer c, c.c_orders o
where c_custkey < 100
order by o_orderpriority, o_orderkey
limit 10;

-- executing against localhost:21000
SELECT t2.s_name
FROM tpch_nested_parquet.customer t1
INNER JOIN tpch_nested_parquet.supplier t2
  ON t2.s_phone = t1.c_phone AND t2.s_acctbal = t1.c_acctbal,
t2.s_partsupps t3
WHERE t2.s_suppkey > t2.s_nationkey;

-- executing against localhost:21000
SELECT STRAIGHT_JOIN r_regionkey, p_retailprice
FROM (SELECT p1.p_partkey, p2.p_retailprice, p2.p_mfgr
      FROM part p1 INNER JOIN part p2 ON p1.p_partkey = p2.p_partkey
      WHERE p1.p_partkey < 100 AND p2.p_partkey < 100) t2
RIGHT JOIN (SELECT r_name, r_regionkey
            FROM region r, r.r_nations
            WHERE r_name = 'AFRICA') t1
  ON t1.r_regionkey = t2.p_partkey;

-- executing against localhost:21000
SELECT COALESCE(t2.p_size, t4.pos, o_orderkey) AS int_col, COUNT(*)
FROM customer t1
LEFT JOIN part t2 ON t1.c_custkey = t2.p_partkey,
t1.c_orders t3,
t3.item.o_lineitems t4
WHERE t1.c_custkey < 100 AND t2.p_partkey < 100
GROUP BY 1
HAVING int_col <= 5
ORDER BY 1;

-- executing against localhost:21000
SELECT o_orderkey, r FROM
  (SELECT o2.o_orderkey, COUNT(o1.pos) OVER (ORDER BY o2.o_orderkey DESC) r
   FROM tpch_nested_parquet.customer c
   JOIN c.c_orders o1
   JOIN c.c_orders o2 ON (o1.pos = o2.pos)) v
ORDER BY o_orderkey LIMIT 1;

-- executing against localhost:21000
set mem_limit=4m;

-- executing against localhost:21000
select max(cnt) from customer c,
(select count(l_returnflag) cnt from c.c_orders.o_lineitems) v;

-- executing against localhost:21000
set num_scanner_threads=1;

-- executing against localhost:21000
set mem_limit=1g;

-- executing against localhost:21000
select *
from tpch_nested_parquet.customer c, c.c_orders o, o.o_lineitems
order by l_partkey desc, l_suppkey desc, l_linenumber desc, c_custkey
limit 5;

Attachments

Issue Links

relates to

IMPALA-2831 Impala can spin up too many scanner threads

Resolved

TestNestedTypes.test_tpch fails with memory limit exceeded

Details

Description

Attachments

Issue Links

Activity

People

Dates