Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-9859

kafka-streams-application-reset tool doesn't take into account topics generated by KTable foreign key join operation

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.6.0
    • Component/s: streams, tools
    • Labels:

      Description

      Steps to reproduce:

      • Create Kafka Streams application which uses foreign key join operation (without a Named parameter overload)
      • Stop Kafka streams application
      • Perform `kafka-topics-list` and verify that foreign key operation internal topics are generated
      • Use `kafka-streams-application-reset` to perform the cleanup of your kafka streams application: `kafka-streams-application-reset --application-id <your_app_id> --input-topics <your_input_topic> --bootstrap-servers <your_bootstrap_server> --to-datetime 2019-04-13T00:00:00.000`
      • Perform `kafka-topics-list` again, you'll see that topics generated by the foreign key operation are still there.

      kafka-streams-application-reset] uses `-subscription-registration-topic` and `-subscription-response-topic` suffixes to match topics generated by the foreign key operation. While in reality, internal topics are generated in this format:

      <application id>-KTABLE-FK-JOIN-SUBSCRIPTION-REGISTRATION-<running number>-topic 
      <application id>-KTABLE-FK-JOIN-SUBSCRIPTION-RESPONSE-<running number>-topic

      Please note that this problem only happens when `Named` parameter is not used. When named parameter is used, topics are generated with a same pattern as specified in StreamsResetter.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                lkokhreidze Levani Kokhreidze
                Reporter:
                lkokhreidze Levani Kokhreidze
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: