Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-1897

Enhance MockProducer for more sophisticated tests

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: producer
    • Labels:
      None

      Description

      Based on the experience of upgrading the kafka producer in Samza, we faced two main constraints when using MockProducer:

      1. The constructor requires a cluster specification and the tools to create a test cluster is not exposed. It is available from TestUtils in Kafka, however that jar is not published. This issue is currently being addressed in KAFKA-1861.
      2. No support for testing a blocking client call. For example, "flush" in Samza blocks on the future returned by the latest send request. In order to test this, the MockProducer which buffers it should run in a concurrent mode. There is currently no provision to do this. We want the MockProducer to buffer the send and then, complete the callback concurrently while we wait for "flush" to unblock.
      We can write unit tests that have improved coverage if we can add support for concurrent execution of the MockProducer and unit test thread. For example implementation, please refer to the latest version of KafkaSystemProducer.scala in the Apache Samza repository.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                junrao Jun Rao
                Reporter:
                navina Navina Ramesh
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: