[IMPALA-6597] Inert into partitioned runs out of memory

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: Impala 2.6.0
• Component/s: Backend
• Epic Color: ghx-label-1
• Assignee: Unassigned
• Reporter: Istvan Szukacs
• Created: 01/Mar/18 10:56
• Updated: 01/Nov/18 01:00
• Resolved: 01/Nov/18 01:00

Description

I am not sure how to interpret the following:

WARNINGS:
Memory limit exceeded
Failed to allocate tuple buffer

Memory Limit Exceeded
 HDFS_SCAN_NODE (id=1) could not allocate 1.42 MB without exceeding limit.
Query(e44f072adac3070c:7cd293d8aa10b8a) Limit: Consumption=12.94 GB
 Fragment e44f072adac3070c:7cd293d8aa10b8e: Consumption=12.94 GB
 SORT_NODE (id=3): Consumption=12.77 GB
 UNION_NODE (id=0): Consumption=2.85 MB
 HDFS_SCAN_NODE (id=1): Consumption=174.29 MB
 HDFS_SCAN_NODE (id=2): Consumption=0
 DataStreamSender: Consumption=88.00 B
 Block Manager: Limit=10.51 GB Consumption=0
Memory Limit Exceeded
Query(e44f072adac3070c:7cd293d8aa10b8a) Limit: Consumption=12.79 GB
 Fragment e44f072adac3070c:7cd293d8aa10b8e: Consumption=12.79 GB
 SORT_NODE (id=3): Consumption=12.79 GB
 UNION_NODE (id=0): Consumption=8.00 KB
 HDFS_SCAN_NODE (id=1): Consumption=124.00 KB
 HDFS_SCAN_NODE (id=2): Consumption=0
 Block Manager: Limit=10.51 GB Consumption=0

mem_limit is configured to be 32GB. What configuration value can be tuned to have more memory for an Impala query?

 

Issue Links

• duplicates: IMPALA-4899 Parquet table writer leaks dictionaries
• relates to: IMPALA-4899 Parquet table writer leaks dictionaries