[SPARK-39575] ByteBuffer forget to rewind after get in AvroDeserializer

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.2.2, 3.3.1, 3.4.0
• Component/s: Spark Core
• Assignee: Frank Wong
• Reporter: Frank Wong
• Created: 24/Jun/22 05:01
• Updated: 27/Jun/22 02:07
• Resolved: 27/Jun/22 02:07

Description

case (BYTES, BinaryType) => (updater, ordinal, value) =>
  val bytes = value match {
    case b: ByteBuffer =>
      val bytes = new Array[Byte](b.remaining)
      b.get(bytes)
      // Do not forget to reset the position
      b.rewind()
      bytes
    case b: Array[Byte] => b
    case other => throw new RuntimeException(s"$other is not a valid avro binary.")
  }
  updater.set(ordinal, bytes) 

After Avro data is converted to InternalRow, there will be redundant position in ByteBuffer type caused by ByteBuffer#get

Issue Links

• links to: [Github] Pull Request #36973 (wzx140)