[IMPALA-7220] Queries processing very large strings hit "memory limit exceeded" instead of spilling

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Backend
• Labels: resource-management
• Environment: m4.4xl, ubuntu 16.04
• Epic Link: Operator memory constraints
• Target Version: Product Backlog
• Epic Color: ghx-label-4
• Assignee: Unassigned
• Reporter: Jim Apple
• Created: 28/Jun/18 23:17
• Updated: 24/Feb/23 03:15

Description

I saw primitive_shuffle_1mb_rows failing at scale factor 4 with 3 impalads running:

[scheduler] [tpch4_parquet Thread 0]: Running Query: primitive_shuffle_1mb_rows
[query_exec_functions] [tpch4_parquet Thread 0]: Connected to localhost:21001
[query_exec_functions] [tpch4_parquet Thread 0]: Connected to localhost:21001
[query_exec_functions] [tpch4_parquet Thread 0]: ImpalaBeeswaxException:
 Query aborted:Memory limit exceeded: Error occurred on backend ip-172-31-25-187:22001 by fragment cd4f016604f24316:c310bea60000000f
Memory left in process limit: -410.47 MB

Command run was

./bin/single_node_perf_run.py --iterations 2 --scale 4 --table_formats parquet/none --workload targeted-perf --num_impalads 3 --query_names '.*' --load --start_minicluster $HASH1 $HASH2

https://jenkins.impala.io/view/Experimental/job/perf-AB-test/182/consoleText

cc: janulatha