Details
-
Bug
-
Status: Closed
-
Minor
-
Resolution: Resolved
-
None
-
None
Description
When executing vector component TestValueVector test cases(testVariableVectorReallocation,testFixedVectorReallocation),exception can be seen.
Exception:
java.lang.Exception: Unexpected exception, expected<org.apache.arrow.vector.util.OversizedAllocationException> but was<org.apache.arrow.memory.OutOfMemoryException>
where the issue comes from:
public void testFixedVectorReallocation() {
....
try {
vector.allocateNew(expectedValueCapacity); <-- where the exception comes from
....
analysis:
allocateNew will invoke classes from io.netty.buffer from memory component and finally invoke netty third party package.I assume there is no issue when invoking memory part.Then I trace that the issue is caused by oversizing value of initialCapacity setting when invoking AbstractByteBufAllocator class in third party netty-buffer-4.0.27.final.jar. I create a test case which can reproduces the issue. value of parameter initialCapacity and maxCapacity are obtained from debugging procedure
public void testAllocator(){
int initialCapacity=2147483644;
int maxCapacity=2147483647;
ByteBuf buf = UnpooledByteBufAllocator.DEFAULT.directBuffer(initialCapacity, maxCapacity);
}
And I checked the rooted affected class java.nio.Bits:
static void reserveMemory(long size, int cap) {
.....
if (totalCapacity + cap > maxMemory)
throw new OutOfMemoryError("Direct buffer memory");
.....
maxMemory can only be set to 1836056576 at most,but cap(initialCapacity) was 2147483644 at this moment.
Attachments
Issue Links
- duplicates
-
ARROW-113 TestValueVector test fails if cannot allocate 2GB of memory
- Resolved