Description
SPARK-10474 (https://github.com/apache/spark/pull/8888) removed the pointer array tracking because `TungstenAggregate` would fail under memory pressure. However, this is somewhat of a hack that we should fix in the right way in 1.6.0 to ensure we don't OOM because of this.
Attachments
Issue Links
- is related to
-
SPARK-10474 TungstenAggregation cannot acquire memory for pointer array after switching to sort-based
- Closed
- relates to
-
SPARK-10929 Tungsten fails to acquire memory writing to HDFS
- Resolved