[KAFKA-13099] Message too large error when expiring transactionalIds

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.1, 2.1.1, 2.2.2, 2.3.1, 2.4.1, 2.5.1, 2.6.1, 2.8.0, 2.7.1
• Fix Version/s: 2.5.2, 2.6.3, 2.7.2, 2.8.1, 3.0.0
• Assignee: Jason Gustafson
• Reporter: Jason Gustafson
• Created: 16/Jul/21 18:21
• Updated: 03/Aug/21 17:25
• Resolved: 28/Jul/21 18:27

Description

We have seen a couple reports of MESSAGE_TOO_LARGE errors when writing tombstones for expired transactionalIds. This is possible because we collect all expired IDs into a single batch. We should ensure that the created batches are smaller than the max message size. Any expired IDs that cannot fit can be expired later.

Issue Links

• links to: GitHub Pull Request #11098