[FLINK-10342] Kafka duplicate topic consumption when topic name is changed

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.8.0
• Component/s: Connectors / Kafka
• Labels: pull-request-available

Release Note

Starting from 1.8.0, the {{FlinkKafkaConsumer}} now always filter out restored partitions that are no longer associated with a specified topic to subscribe to in the restored execution. This behaviour did not exist in previous versions of the {{FlinkKafkaConsumer}}. If you wish to retain the previous behaviour, please use the {{disableFilterRestoredPartitionsWithSubscribedTopics}} configuration method on the {{FlinkKafkaConsumer}}.

• Assignee: Feng LI
• Reporter: Oleksandr Nitavskyi
• Created: 13/Sep/18 14:50
• Updated: 28/Feb/19 11:20
• Resolved: 28/Feb/19 11:20

Description

In case of topic name is simply renamed for a KafkaConsumer Flink starts to consume from old and a new topic in the same time which can lead to unexpected behavior.

Here is the PR with reproduce: https://github.com/apache/flink/pull/6691

 

Issue Links

• is duplicated by: FLINK-10807 KafkaConsumer still consume removed topic after changing topics list
• links to: GitHub Pull Request #7726