[FLINK-27480] KafkaSources sharing the groupId might lead to InstanceAlreadyExistException warning

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.14.4, 1.15.0, 1.16.0
• Fix Version/s: 1.14.5, 1.15.1, 1.16.0
• Component/s: Connectors / Kafka
• Labels: pull-request-available
• Assignee: Fabian Paul
• Reporter: Fabian Paul
• Created: 03/May/22 08:19
• Updated: 09/May/22 08:58
• Resolved: 09/May/22 08:58

Description

More and more frequently, users ran into an issue by not correctly configuring the KafkaSource (https://stackoverflow.com/questions/72026997/flink-instancealreadyexistsexception-while-migrating-to-the-kafkasource) and setting non-distinguishable groupIds for the source. Internally the used KafkaConsumer tries to register with the metric system and incorporates the groupId as part of the metric, leading to name collision.

We should update the documentation to explain the situation properly.

Issue Links

• links to: GitHub Pull Request #19629
• links to: GitHub Pull Request #19658
• links to: GitHub Pull Request #19659