[KAFKA-13406] Cooperative sticky assignor got stuck due to assignment validation failed

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.1.0, 3.0.1, 2.8.2
• Component/s: clients
• Assignee: Luke Chen
• Reporter: Luke Chen
• Created: 27/Oct/21 07:32
• Updated: 23/Feb/22 01:41
• Resolved: 16/Nov/21 03:09

Description

We'll do validateCooperativeAssignment for cooperative assignor, where we validate if there are previously owned partitions directly transfer to other consumers without "revoke" step. However, the "ownedPartition" in subscription might contain out-of-dated data, which might cause the validation always failure.

We should consider the short-term fix it by disabling validateCooperationAssignment for built-in cooperativeStickyAssignor because we've already consider the generation in the assignor, and discard the old generation ownedPartition if any.

Issue Links

• is caused by: KAFKA-13419 sync group failed with rebalanceInProgress error might cause out-of-date ownedPartition in Cooperative protocol
• relates to: KAFKA-12984 Cooperative sticky assignor can get stuck with invalid SubscriptionState input metadata
• links to: GitHub Pull Request #11439