[KAFKA-8575] Investigate removing EAGER protocol & cleaning up task suspension in Streams rebalancing

• Type: Improvement
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 2.4.0
• Fix Version/s: 4.0.0
• Component/s: streams
• Assignee: Unassigned
• Reporter: A. Sophie Blee-Goldman
• Created: 20/Jun/19 22:59
• Updated: 23/Nov/23 01:14

Description

With KIP-429 the suspend/resume of tasks may have minimal gains while adding a lot of complexity and potential bugs. We should consider removing/cleaning it up and going a step further to remove the EAGER protocol from Streams entirely.

Plan to remove this in 3.1/4.0, whichever comes after 3.0. This will make 3.0 a bridge release for users upgrading from any version below 2.4, but they will still be able to do so in the usual two rolling bounces.

 

The upgrade path from 2.3 and below, to any {to_version} higher than 3.1 will be:

1. During the first rolling bounce, upgrade the jars to a version between 2.4 - 3.1 and add the UPGRADE_FROM config for whichever version you are upgrading from
2. During the second rolling bounce, upgrade the jars to the desired {to_version} and remove the UPGRADE_FROM config

 

EAGER will be effectively deprecated in 3.0 but not removed until the next version.

Issue Links

• is blocked by: KAFKA-13439 Deprecate EAGER rebalancing in Kafka Streams
• links to: Github Pull Request #7321