Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-8233

KIP-470: TopologyTestDriver test input and output usability improvements

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 2.4.0
    • streams

    Description

      When using TopologyTestDriver you need to call ConsumerRecordFactory to create ConsumerRecord passed into pipeInput method to write to topic. Also when calling readOutput to consume from topic, you need to provide correct Deserializers each time.

      You easily end up writing helper methods in your test classed, but this can be avoided when adding generic input and output topic classes.

      This improvement adds TestInputTopic class which wraps TopologyTestDriver  and ConsumerRecordFactory methods as one class to be used to write to Input Topics and TestOutputTopic class which collects TopologyTestDriver  reading methods and provide typesafe read methods.

      This improvement deprecates the old TopologyTestDriver input and output methods in favor of TestInputTopic  and TestOutputTopic classes. Also now obsoleted ConsumerRecordFactory  and OutputVerifier classes are deprecated.

      This is KIP:

      https://cwiki.apache.org/confluence/display/KAFKA/KIP-470%3A+TopologyTestDriver+test+input+and+output+usability+improvements 

       

      More info and an example of how Stream test looks after using this classes:

      https://github.com/jukkakarvanen/kafka-streams-test-topics

      Attachments

        Issue Links

          Activity

            Thanks for creating the ticket. If you want to contribute something like this, it would be required to write a KIP: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals

            mjsax Matthias J. Sax added a comment - Thanks for creating the ticket. If you want to contribute something like this, it would be required to write a KIP:  https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
            githubbot ASF GitHub Bot added a comment -

            jukkakarvanen commented on pull request #7378: KAFKA-8233: [WIP] KIP-470: TopologyTestDriver test input and output usability improvements
            URL: https://github.com/apache/kafka/pull/7378

            This is WIP PR for KIP-470: TopologyTestDriver test input and output usability improvements:
            https://cwiki.apache.org/confluence/display/KAFKA/KIP-470%3A+TopologyTestDriver+test+input+and+output+usability+improvements

            Implementation mainly done.

            Still to do:
            -Check unit test coverage
            -Update example application
            -Update documentation

                1. Committer Checklist (excluded from commit message)
            • [ ] Verify design and implementation
            • [ ] Verify test coverage and CI build status
            • [ ] Verify documentation (including upgrade notes)

            ----------------------------------------------------------------
            This is an automated message from the Apache Git Service.
            To respond to the message, please log on to GitHub and use the
            URL above to go to the specific comment.

            For queries about this service, please contact Infrastructure at:
            users@infra.apache.org

            githubbot ASF GitHub Bot added a comment - jukkakarvanen commented on pull request #7378: KAFKA-8233 : [WIP] KIP-470: TopologyTestDriver test input and output usability improvements URL: https://github.com/apache/kafka/pull/7378 This is WIP PR for KIP-470: TopologyTestDriver test input and output usability improvements: https://cwiki.apache.org/confluence/display/KAFKA/KIP-470%3A+TopologyTestDriver+test+input+and+output+usability+improvements Implementation mainly done. Still to do: -Check unit test coverage -Update example application -Update documentation Committer Checklist (excluded from commit message) [ ] Verify design and implementation [ ] Verify test coverage and CI build status [ ] Verify documentation (including upgrade notes) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org

            jkarvanen – I added you to the list of contributors and assigned the ticket to you. You can now also self-assign ticket.

            mjsax Matthias J. Sax added a comment - jkarvanen – I added you to the list of contributors and assigned the ticket to you. You can now also self-assign ticket.
            githubbot ASF GitHub Bot added a comment -

            mjsax commented on pull request #7378: KAFKA-8233: KIP-470: TopologyTestDriver test input and output usability improvements
            URL: https://github.com/apache/kafka/pull/7378

            ----------------------------------------------------------------
            This is an automated message from the Apache Git Service.
            To respond to the message, please log on to GitHub and use the
            URL above to go to the specific comment.

            For queries about this service, please contact Infrastructure at:
            users@infra.apache.org

            githubbot ASF GitHub Bot added a comment - mjsax commented on pull request #7378: KAFKA-8233 : KIP-470: TopologyTestDriver test input and output usability improvements URL: https://github.com/apache/kafka/pull/7378 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org

            People

              jkarvanen Jukka Karvanen
              jkarvanen Jukka Karvanen
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: