[KAFKA-9184] Redundant task creation and periodic rebalances after zombie worker rejoins the group

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.4.0, 2.3.2
• Fix Version/s: 2.4.0, 2.5.0, 2.3.2
• Component/s: KafkaConnect
• Assignee: Konstantine Karantasis
• Reporter: Konstantine Karantasis
• Reviewer: Randall Hauch
• Created: 14/Nov/19 01:53
• Updated: 04/Mar/20 20:58
• Resolved: 05/Dec/19 02:01

Description

First reported here:

https://stackoverflow.com/questions/58631092/kafka-connect-assigns-same-task-to-multiple-workers

There seems to be an issue with task reassignment when a worker rejoins after an unsuccessful join request. The worker seems to be outside the group for a generation but when it joins again the same task is running in more than one worker

Issue Links

• is related to: KAFKA-9385 Connect cluster: connector task repeat like a splitbrain cluster problem
• is related to: KAFKA-9653 Duplicate tasks on workers after rebalance
• links to: GitHub Pull Request #7771
• links to: GitHub Pull Request #7783