[KAFKA-1897] Enhance MockProducer for more sophisticated tests

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: producer
• Assignee: Jun Rao
• Reporter: Navina Ramesh
• Created: 24/Jan/15 01:48
• Updated: 21/Aug/15 02:16

Description

Based on the experience of upgrading the kafka producer in Samza, we faced two main constraints when using MockProducer:

1. The constructor requires a cluster specification and the tools to create a test cluster is not exposed. It is available from TestUtils in Kafka, however that jar is not published. This issue is currently being addressed in KAFKA-1861.
2. No support for testing a blocking client call. For example, "flush" in Samza blocks on the future returned by the latest send request. In order to test this, the MockProducer which buffers it should run in a concurrent mode. There is currently no provision to do this. We want the MockProducer to buffer the send and then, complete the callback concurrently while we wait for "flush" to unblock.
We can write unit tests that have improved coverage if we can add support for concurrent execution of the MockProducer and unit test thread. For example implementation, please refer to the latest version of KafkaSystemProducer.scala in the Apache Samza repository.

Issue Links

• blocks: SAMZA-523 Remove custom implementation of MockKafkaProducer, once the missing features are added to MockProducer in Kafka