Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
The current MessageSerializer implementation is wasteful when used to read an IPC payload that is already in-memory in an ArrowBuf. In particular, reads out of a ReadChannel require memory allocation
In C++, we have abstracted memory allocation out of the IPC read path so that zero-copy is possible. I suggest that a similar mechanism can be developed for Java to improve deserialization performance for in-memory messages. The new interface would return ArrowBuf when performing reads, which could be zero-copy when possible, but when not the current strategy of allocate-copy could be used