[KAFKA-14996] The KRaft controller should properly handle overly large user operations

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.5.0
• Fix Version/s: 3.5.0
• Component/s: controller
• Assignee: Colin McCabe
• Reporter: Edoardo Comar
• Created: 12/May/23 18:30
• Updated: 02/Jun/23 14:57
• Resolved: 02/Jun/23 14:57

Description

If an attempt is made to create a topic with

num partitions >= QuorumController.MAX_RECORDS_PER_BATCH  (10000)

the client receives an UnknownServerException - it could rather receive a better error.

The controller logs

2023-05-12 19:25:10,018] WARN [QuorumController id=1] createTopics: failed with unknown server exception IllegalStateException at epoch 2 in 21956 us.  Renouncing leadership and reverting to the last committed offset 174. (org.apache.kafka.controller.QuorumController)
java.lang.IllegalStateException: Attempted to atomically commit 10001 records, but maxRecordsPerBatch is 10000
    at org.apache.kafka.controller.QuorumController.appendRecords(QuorumController.java:812)
    at org.apache.kafka.controller.QuorumController$ControllerWriteEvent.run(QuorumController.java:719)
    at org.apache.kafka.queue.KafkaEventQueue$EventContext.run(KafkaEventQueue.java:127)
    at org.apache.kafka.queue.KafkaEventQueue$EventHandler.handleEvents(KafkaEventQueue.java:210)
    at org.apache.kafka.queue.KafkaEventQueue$EventHandler.run(KafkaEventQueue.java:181)
    at java.base/java.lang.Thread.run(Thread.java:829)
[

Issue Links

• links to: GitHub Pull Request #13742
• links to: GitHub Pull Request #13761
• links to: GitHub Pull Request #13766