[PARQUET-1417] BINARY_AS_SIGNED_INTEGER_COMPARATOR fails with IOBE for the same arrays with the different length

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.10.0
• Fix Version/s: 1.11.0
• Labels: pull-request-available
• Assignee: Vova Vysotskyi
• Reporter: Vova Vysotskyi
• Created: 14/Sep/18 16:44
• Updated: 21/Sep/18 15:12
• Resolved: 20/Sep/18 09:36

Description

BINARY_AS_SIGNED_INTEGER_COMPARATOR fails when the same byte arrays but with the different number leading zeros are compared:

    BINARY_AS_SIGNED_INTEGER_COMPARATOR.compare(
        Binary.fromConstantByteBuffer(ByteBuffer.wrap(new byte[] { 0, 0, -108 })),
        Binary.fromConstantByteBuffer(ByteBuffer.wrap(new byte[] { 0, -108 })));

Error is:

java.lang.IndexOutOfBoundsException
	at java.nio.Buffer.checkIndex(Buffer.java:540)
	at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:139)
	at org.apache.parquet.schema.PrimitiveComparator$9.compare(PrimitiveComparator.java:280)
	at org.apache.parquet.schema.PrimitiveComparator$9.compare(PrimitiveComparator.java:262)
	at org.apache.parquet.schema.PrimitiveComparator$BinaryComparator.compareNotNulls(PrimitiveComparator.java:186)
	at org.apache.parquet.schema.PrimitiveComparator$BinaryComparator.compareNotNulls(PrimitiveComparator.java:183)
	at org.apache.parquet.schema.PrimitiveComparator.compare(PrimitiveComparator.java:63)

The problem is that BINARY_AS_SIGNED_INTEGER_COMPARATOR.compare(ByteBuffer b1, ByteBuffer b2) method passes the length of the first ByteBuffer, but it should pass the less length since padding was calculated and passed for the ByteBuffer with greater length to the compare(int length, ByteBuffer b1, int p1, ByteBuffer b2, int p2) method.

Issue Links

• links to: GitHub Pull Request #522