[ARROW-414] [Java] "Buffer too large to resize to ..." error

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.2.0
• Component/s: Java
• External issue URL: https://github.com/apache/arrow/issues/16063
• Assignee: Wes McKinney
• Reporter: Wes McKinney
• Created: 09/Dec/16 18:44
• Updated: 11/Jan/23 07:08
• Resolved: 10/Dec/16 00:51

Description

On the latest branch for ARROW-394 (after rebasing on all the fixes so far), I have the error

Incompatible files
Could not load buffers for field f2: Utf8. error message: Buffer too large to resize to 44: 128
13:41:19.356 [main] ERROR org.apache.arrow.tools.Integration - Incompatible files
java.lang.IllegalArgumentException: Could not load buffers for field f2: Utf8. error message: Buffer too large to resize to 44: 128
	at org.apache.arrow.vector.VectorLoader.loadBuffers(VectorLoader.java:84) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.apache.arrow.vector.VectorLoader.loadBuffers(VectorLoader.java:94) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.apache.arrow.vector.VectorLoader.load(VectorLoader.java:63) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.apache.arrow.tools.Integration$Command$3.execute(Integration.java:156) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.apache.arrow.tools.Integration.run(Integration.java:212) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.apache.arrow.tools.Integration.main(Integration.java:61) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
Caused by: java.lang.IllegalArgumentException: Buffer too large to resize to 44: 128
	at org.apache.arrow.vector.BaseDataValueVector.truncateBufferBasedOnSize(BaseDataValueVector.java:55) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.apache.arrow.vector.NullableVarCharVector.loadFieldBuffers(NullableVarCharVector.java:142) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.apache.arrow.vector.VectorLoader.loadBuffers(VectorLoader.java:82) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
	... 5 common frames omitted

See attached JSON and C++-generated Arrow file