Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-8575

Investigate removing EAGER protocol & cleaning up task suspension in Streams rebalancing

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Critical
    • Resolution: Unresolved
    • 2.4.0
    • 4.0.0
    • streams
    • None

    Description

      With KIP-429 the suspend/resume of tasks may have minimal gains while adding a lot of complexity and potential bugs. We should consider removing/cleaning it up and going a step further to remove the EAGER protocol from Streams entirely.

      Plan to remove this in 3.1/4.0, whichever comes after 3.0. This will make 3.0 a bridge release for users upgrading from any version below 2.4, but they will still be able to do so in the usual two rolling bounces.

       

      The upgrade path from 2.3 and below, to any {to_version} higher than 3.1 will be:

      1. During the first rolling bounce, upgrade the jars to a version between 2.4 - 3.1 and add the UPGRADE_FROM config for whichever version you are upgrading from
      2. During the second rolling bounce, upgrade the jars to the desired {to_version} and remove the UPGRADE_FROM config

       

      EAGER will be effectively deprecated in 3.0 but not removed until the next version.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            ableegoldman A. Sophie Blee-Goldman

            Dates

              Created:
              Updated:

              Slack

                Issue deployment