[ARROW-5821] [Java] Support compact fixed-width vectors

• Type: New Feature
• Status: Closed
• Priority: Minor
• Resolution: Invalid
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/16795
• Assignee: Ji Liu
• Reporter: Ji Liu
• Created: 02/Jul/19 08:36
• Updated: 11/Jan/23 07:42
• Resolved: 03/Sep/19 08:26

Description

In shuffle stage of some applications, FixedWitdhVectors may have very little non-null data.
In this case, directly serialize vectors is not a good choice, generally we can compact the vector make it only holding non-null value and create a BitVector to trace the indices for non-null values so that it could be deserialized properly.

Issue Links

• relates to: ARROW-300 [Format] Add body buffer compression option to IPC message protocol using LZ4 or ZSTD
• links to: GitHub Pull Request #4789