[KAFKA-8570] Downconversion could fail when log contains out of order message formats

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.1.2, 2.0.2, 2.1.2, 2.2.2, 2.3.1
• Component/s: clients
• Assignee: Dhruvil Shah
• Reporter: Dhruvil Shah
• Created: 20/Jun/19 00:17
• Updated: 11/Sep/19 14:42
• Resolved: 21/Jun/19 16:46

Description

When the log contains out of order message formats (for example a v2 message followed by a v1 message), it is possible for down-conversion to fail in certain scenarios where batches compressed and greater than 1kB in size. Down-conversion fails with a stack like the following:

java.lang.IllegalArgumentException
at java.nio.Buffer.limit(Buffer.java:275)
at org.apache.kafka.common.record.FileLogInputStream$FileChannelRecordBatch.writeTo(FileLogInputStream.java:176)
at org.apache.kafka.common.record.AbstractRecords.downConvert(AbstractRecords.java:107)
at org.apache.kafka.common.record.FileRecords.downConvert(FileRecords.java:242)

Issue Links

• links to: GitHub Pull Request #6974
• links to: GitHub Pull Request #7071