Details
-
Bug
-
Status: Resolved
-
Trivial
-
Resolution: Duplicate
-
Impala 2.0
-
None
-
None
Description
When running out of memory in PAGG the error message does not make it
clear why we ran out of memory. In the example below we see that the
limit is 100MB and the consumption is only 8MB, but we ran out of memory.
In this particular case, the reason is that PAGG tries to allocate the
initial min mem for each partition and fails (each partition needs ~8MB
and we have 32 of them).
It would be good to improve the error message with some more useful info,
e.g. how much more memory was needed.
Backend 0:Memory Limit Exceeded
Query(3b428a0f1e22f611:722a79f624fcb0b5) Limit: Limit=100.00 MB Consumption=8.03 MB
Fragment 3b428a0f1e22f611:722a79f624fcb0b6: Consumption=8.00 KB
EXCHANGE_NODE (id=5): Consumption=0
DataStreamRecvr: Consumption=0
Block Manager: Consumption=0
Fragment 3b428a0f1e22f611:722a79f624fcb0b7: Consumption=4.01 MB
SORT_NODE (id=2): Consumption=0
AGGREGATION_NODE (id=4): Consumption=4.00 MB
EXCHANGE_NODE (id=3): Consumption=0
DataStreamRecvr: Consumption=0
DataStreamSender: Consumption=4.00 KB
Fragment 3b428a0f1e22f611:722a79f624fcb0b8: Consumption=4.01 MB
AGGREGATION_NODE (id=1): Consumption=4.00 MB
HDFS_SCAN_NODE (id=0): Consumption=0
DataStreamSender: Consumption=4.00 KB