Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Cannot Reproduce
-
2.4.3
-
None
-
None
Description
Currently, bufferPoolsBySize in HeapMemoryAllocator uses a Map type whose value type is LinkedList.
LinkedList is not thread safe and may hit the error below
java.util.NoSuchElementExceptionException at java.util.LinkedList.removeFirst(LinkedList.java:270) at java.util.LinkedList.remove(LinkedList.java:685) at org.apache.spark.unsafe.memory.HeapMemoryAllocator.allocate(HeapMemoryAllocator.java:57)