Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-1417

BINARY_AS_SIGNED_INTEGER_COMPARATOR fails with IOBE for the same arrays with the different length

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.10.0
    • Fix Version/s: 1.11.0
    • Component/s: None

      Description

      BINARY_AS_SIGNED_INTEGER_COMPARATOR fails when the same byte arrays but with the different number leading zeros are compared:

          BINARY_AS_SIGNED_INTEGER_COMPARATOR.compare(
              Binary.fromConstantByteBuffer(ByteBuffer.wrap(new byte[] { 0, 0, -108 })),
              Binary.fromConstantByteBuffer(ByteBuffer.wrap(new byte[] { 0, -108 })));
      

      Error is:

      java.lang.IndexOutOfBoundsException
      	at java.nio.Buffer.checkIndex(Buffer.java:540)
      	at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:139)
      	at org.apache.parquet.schema.PrimitiveComparator$9.compare(PrimitiveComparator.java:280)
      	at org.apache.parquet.schema.PrimitiveComparator$9.compare(PrimitiveComparator.java:262)
      	at org.apache.parquet.schema.PrimitiveComparator$BinaryComparator.compareNotNulls(PrimitiveComparator.java:186)
      	at org.apache.parquet.schema.PrimitiveComparator$BinaryComparator.compareNotNulls(PrimitiveComparator.java:183)
      	at org.apache.parquet.schema.PrimitiveComparator.compare(PrimitiveComparator.java:63)
      

      The problem is that BINARY_AS_SIGNED_INTEGER_COMPARATOR.compare(ByteBuffer b1, ByteBuffer b2) method passes the length of the first ByteBuffer, but it should pass the less length since padding was calculated and passed for the ByteBuffer with greater length to the compare(int length, ByteBuffer b1, int p1, ByteBuffer b2, int p2) method.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                vvysotskyi Volodymyr Vysotskyi
                Reporter:
                vvysotskyi Volodymyr Vysotskyi
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: