Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
None
-
None
-
None
Description
Problem: Jvm crash in Load scenario when unsafe memory allocation is failed.
scenario:
a) Have many cores while loading. suggested more than 10. [carbon.number.of.cores.while.loading]
b) Load huge data with local sort, more than 5GB (keeping default unsafe memory manager as 512 MB)
c) when task failes due to not enough unsafae memory, JVM crashes with SIGSEGV.
root casue:
while sorting, all iterator threads are waiting at UnsafeSortDataRows.addRowBatch as all iterator works on one row page.
Only one iterator thread will try to allocate memory. Before that it has freed current page in handlePreviousPage().
When allocate memory failed, row page will still have that old reference. next thread will again use same reference and call handlePreviousPage().
So, Jvm crashes as freed memory is accessed.
solution:
When allocation failed, set row page reference to null.
So, that next thread will not do any operation.
Attachments
Issue Links
- links to
