Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-16758

Extend Consumer#close with option to leave the group or not

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments


    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • 4.0.0
    • clients, consumer


      See comments on https://issues.apache.org/jira/browse/KAFKA-16514 for the full context.

      Essentially we would get rid of the "internal.leave.group.on.close" config that is used as a backdoor by Kafka Streams right now to prevent closed consumers from leaving the group, thus reducing unnecessary task movements after a simple bounce. 

      This would be replaced by an actual public API that would allow the caller to opt in or out to the LeaveGroup when close is called. This would be similar to the KafkaStreams#close(CloseOptions) API, and in fact would be how that API will be implemented (since it only works for static groups at the moment as noted in KAFKA-16514 )

      This has several benefits over the current situation:

      1. It allows plain consumer apps to opt-out of leaving the group when closed, which is currently not possible through any public API (only an internal backdoor config)
      2. It enables the caller to dynamically select the appropriate action depending on why the client is being closed – for example, you would not want the consumer to leave the group during a simple restart, but would want it to leave the group when shutting down the app or if scaling down the node. This is not possible today, even with the internal config, since configs are immutable
      3. It can be leveraged to "fix" the KafkaStreams#close(closeOptions) API so that the user's choice to leave the group during close will be respected for non-static members


        Issue Links


          This comment will be Viewable by All Users Viewable by All Users


            lianetm Lianet Magrans
            ableegoldman A. Sophie Blee-Goldman




                Issue deployment