[ARROW-399] [Java] ListVector.loadFieldBuffers ignores the ArrowFieldNode length metadata

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 0.2.0
• Component/s: Java
• External issue URL: https://github.com/apache/arrow/issues/16027
• Assignee: Julien Le Dem
• Reporter: Wes McKinney
• Created: 01/Dec/16 19:53
• Updated: 11/Jan/23 07:08
• Resolved: 09/Dec/16 16:41

Description

Discovered this during integration testing. Because Arrow-C++ writes buffers padded to 64 bytes, they may appear larger to the Java library than they need to be. In ListVector.loadFieldBuffers, the ArrowFieldNode is never used:

  @Override
  public void loadFieldBuffers(ArrowFieldNode fieldNode, List<ArrowBuf> ownBuffers) {
    BaseDataValueVector.load(getFieldInnerVectors(), ownBuffers);
  }

The value count of the resulting ListVector is thus inferred from the size of the offsets buffer. In the case of a length-7 vector in C++, the size of the offsets buffer is exactly 64 bytes (padding for SIMD) – Java infers from 64 bytes that the value count is 15 (64 / 4 - 1), and the integration test fails.

Issue Links

• links to: PR