[IMPALA-2861] 5.4.x-nightly-data-load: test_scanners: Memory Limit Exceeded

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 2.2.10
• Component/s: Backend
• Target Version: Impala 2.2.10
• Assignee: Juan Yu
• Reporter: Daniel Hecht
• Created: 16/Jan/16 06:29
• Updated: 08/Feb/16 16:57
• Resolved: 08/Feb/16 16:57

Description

http://sandbox.jenkins.cloudera.com/job/impala-cdh5-2.2.0_5.4.x-nightly-data-load/189/testReport/junit/query_test.test_scanners/TestScannersAllTableFormats/test_scanners_batch_size__1___exec_option____disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0___batch_size___0___num_nodes___0____table_format__parquet_none_/

E   ImpalaBeeswaxException: ImpalaBeeswaxException:
E    Query aborted:
E   Memory limit exceeded
E   
E   
E   
E   Memory Limit Exceeded
E   Query(5b4e5d9da153dea4:1985436140f29e93) Limit: memory limit exceeded. Limit=5.00 GB Consumption=5.00 GB
E     Fragment 5b4e5d9da153dea4:1985436140f29e97: Consumption=5.00 GB
E       SORT_NODE (id=5): Consumption=4.00 KB
E       CROSS_JOIN_NODE (id=4): Consumption=5.00 GB
E       CROSS_JOIN_NODE (id=3): Consumption=16.00 KB
E       HDFS_SCAN_NODE (id=2): Consumption=0
E       EXCHANGE_NODE (id=6): Consumption=0
E       DataStreamRecvr: Consumption=0
E       EXCHANGE_NODE (id=7): Consumption=0
E       DataStreamRecvr: Consumption=3.32 MB
E       DataStreamSender: Consumption=4.00 KB
E     Block Manager: Limit=4.00 GB Consumption=0
Standard Error

-- connecting to: localhost:21000
-- executing against localhost:21000
use functional_parquet;

SET disable_codegen=False;
SET abort_on_error=1;
SET exec_single_node_rows_threshold=0;
SET batch_size=1;
SET num_nodes=0;
-- executing against localhost:21000
select count(*),
  sum(id), count(bool_col), sum(tinyint_col), sum(smallint_col),
  sum(int_col), sum(bigint_col), max(float_col), max(double_col),
  max(date_string_col), max(string_col), max(timestamp_col)
from alltypesagg
where id % 2 = 0 and day is not null;

-- executing against localhost:21000
select sum(t1.id), sum(t1.int_col),max(t1.date_string_col), max(t2.string_col)
from alltypesagg t1
inner join alltypesagg t2
  on t1.id = t2.id and t1.day is not null and t2.day is not null;

-- executing against localhost:21000
select id, bool_col, int_col
from alltypesagg where day is not null
order by 1 desc, 2 desc, 3 desc
limit 10;

Started failing after this backport

IMPALA-2473: reduce scanner memory usage