[KAFKA-9173] StreamsPartitionAssignor assigns partitions to only one worker

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.3.0, 2.2.1
• Fix Version/s: 2.6.0
• Component/s: streams
• Labels: user-experience
• Assignee: Unassigned
• Reporter: Oleg Muravskiy
• Created: 12/Nov/19 09:17
• Updated: 09/Mar/21 09:18
• Resolved: 22/May/20 11:48

Description

I'm running a distributed KafkaStreams application on 10 worker nodes, subscribed to 21 topics with 10 partitions in each. I'm only using a Processor interface, and a persistent state store.

However, only one worker gets assigned partitions, all other workers get nothing. Restarting the application, or cleaning local state stores does not help. StreamsPartitionAssignor migrates to other nodes, and eventually picks up other node to assign partitions to, but still only one node.

It's difficult to figure out where to look for the signs of problems, I'm attaching the log messages from the StreamsPartitionAssignor. Let me know what else I could provide to help resolve this.

StreamsPartitionAssignor.log