Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-16758

Extend Consumer#close with option to leave the group or not

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • 4.0.0
    • clients, consumer

    Description

      See comments on https://issues.apache.org/jira/browse/KAFKA-16514 for the full context.

      Essentially we would get rid of the "internal.leave.group.on.close" config that is used as a backdoor by Kafka Streams right now to prevent closed consumers from leaving the group, thus reducing unnecessary task movements after a simple bounce. 

      This would be replaced by an actual public API that would allow the caller to opt in or out to the LeaveGroup when close is called. This would be similar to the KafkaStreams#close(CloseOptions) API, and in fact would be how that API will be implemented (since it only works for static groups at the moment as noted in KAFKA-16514 )

      This has several benefits over the current situation:

      1. It allows plain consumer apps to opt-out of leaving the group when closed, which is currently not possible through any public API (only an internal backdoor config)
      2. It enables the caller to dynamically select the appropriate action depending on why the client is being closed – for example, you would not want the consumer to leave the group during a simple restart, but would want it to leave the group when shutting down the app or if scaling down the node. This is not possible today, even with the internal config, since configs are immutable
      3. It can be leveraged to "fix" the KafkaStreams#close(closeOptions) API so that the user's choice to leave the group during close will be respected for non-static members

      Attachments

        Issue Links

          Activity

            People

              lianetm Lianet Magrans
              ableegoldman A. Sophie Blee-Goldman
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: