[KAFKA-8180] Deleting large number of topics can block the controller for the time it takes to delete all of them

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: controller
• Assignee: Unassigned
• Reporter: Gwen Shapira
• Created: 01/Apr/19 19:57
• Updated: 10/Jun/20 14:59

Description

Scenario:

• Create large number of topics (In my experiment: 400 topics with 12 partitions each )
• Use the admin client to delete all of them in a single batch operation
• Try to bounce another broker while this is going on

As you can see from the logs and metrics - topic deletion happens synchronously in the controller and it does not do anything else (leader elections for instance) while it is busy deleting (which can take many minutes for large batches).

I recommend fixing it by throttling the deletes - no matter how large a batch the client sent, the controller should delete a subset and complete a full cycle before deleting the next subset.