Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-10342

Kafka duplicate topic consumption when topic name is changed

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Release Note:
      Hide
      Starting from 1.8.0, the {{FlinkKafkaConsumer}} now always filter out restored partitions that are no longer associated with a specified topic to subscribe to in the restored execution. This behaviour did not exist in previous versions of the {{FlinkKafkaConsumer}}. If you wish to retain the previous behaviour, please use the {{disableFilterRestoredPartitionsWithSubscribedTopics}} configuration method on the {{FlinkKafkaConsumer}}.
      Show
      Starting from 1.8.0, the {{FlinkKafkaConsumer}} now always filter out restored partitions that are no longer associated with a specified topic to subscribe to in the restored execution. This behaviour did not exist in previous versions of the {{FlinkKafkaConsumer}}. If you wish to retain the previous behaviour, please use the {{disableFilterRestoredPartitionsWithSubscribedTopics}} configuration method on the {{FlinkKafkaConsumer}}.

      Description

      In case of topic name is simply renamed for a KafkaConsumer Flink starts to consume from old and a new topic in the same time which can lead to unexpected behavior.

      Here is the PR with reproduce: https://github.com/apache/flink/pull/6691

       

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              f.li Feng LI
              Reporter:
              Oleksandr Nitavskyi Oleksandr Nitavskyi

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 10m
                10m

                  Issue deployment