Major Description
Cooperative rebalancing has been the default since 2.4, but we have always had to keep the logic for eager rebalancing around to allow users a live upgrade path. The current upgrade path involves two rolling bounces, the first one to upgrade the byte code and set the UPGRADE_FROM config to keep Kafka Streams on the old EAGER protocol until everyone has been upgraded, and a second rolling bounce to remove the config and start enabling COOPERATIVE
We'd like to finally remove the EAGER protocol and tackle some tech debt its presence has accrued, but we should first give users a warning that we intend to remove this and that it will require a slight change to the upgrade path for any users who want to upgrade from 2.3 or below: going through a "bridge" version between 2.4 - 3.1 in the first rolling bounce, before upgrading to the final version.
We should also prepare by logging a warning in 3.1 if we see the UPGRADE_FROM config set, informing them that they will need to make sure to remove it before the EAGER protocol is removed. Then in version 3.2 (or whenever we remove it) we still throw an exception and shut down if a user has set the UPGRADE_FROM flag to a pre-2.4 version.
Attachments
Issue Links
- blocks
-
KAFKA-8575 Investigate removing EAGER protocol & cleaning up task suspension in Streams rebalancing
-
- Open
-
- links to