Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-10342

Kafka duplicate topic consumption when topic name is changed

    Details

    • Release Note:
      Hide
      Starting from 1.8.0, the {{FlinkKafkaConsumer}} now always filter out restored partitions that are no longer associated with a specified topic to subscribe to in the restored execution. This behaviour did not exist in previous versions of the {{FlinkKafkaConsumer}}. If you wish to retain the previous behaviour, please use the {{disableFilterRestoredPartitionsWithSubscribedTopics}} configuration method on the {{FlinkKafkaConsumer}}.
      Show
      Starting from 1.8.0, the {{FlinkKafkaConsumer}} now always filter out restored partitions that are no longer associated with a specified topic to subscribe to in the restored execution. This behaviour did not exist in previous versions of the {{FlinkKafkaConsumer}}. If you wish to retain the previous behaviour, please use the {{disableFilterRestoredPartitionsWithSubscribedTopics}} configuration method on the {{FlinkKafkaConsumer}}.

      Description

      In case of topic name is simply renamed for a KafkaConsumer Flink starts to consume from old and a new topic in the same time which can lead to unexpected behavior.

      Here is the PR with reproduce: https://github.com/apache/flink/pull/6691

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                f.li Feng LI
                Reporter:
                Oleksandr Nitavskyi Oleksandr Nitavskyi
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 10m
                  10m