[ARROW-5929] [Java] Define API for ExtensionVector whose data must be serialized prior to being sent via IPC

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Java
• Labels: pull-request-available
• Assignee: Liya Fan
• Reporter: Wes McKinney
• Created: 12/Jul/19 14:30
• Updated: 15/Aug/19 05:06

Description

As being discussed on the mailing list, a possible use case for ExtensionVector involves having the Arrow buffers contain pointer-type values referring to memory outside of the Arrow memory heap. In IPC, such vectors would need to be serialized to a wholly Arrow-resident form, such as a VarBinaryVector. We do not have an API to allow for this, so this JIRA proposes to add new functions that can indicate to the IPC layer that an ExtensionVector requires additional serialization to a native Arrow type (in such case, the extension type metadata would be discarded)

Issue Links

• is related to: ARROW-5931 [C++] Extend extension types facility to provide for serialization and deserialization in IPC roundtrips
• links to: GitHub Pull Request #4889