[KAFKA-7599] Trogdor - Allow configuration for not throttling Benchmark Workers and expose messages per second in task status

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Stanislav Kozlovski
• Created: 06/Nov/18 14:37
• Updated: 02/Jun/20 16:43

Description

In Trogdor, the ConsumeBench, ProduceBench and RoundTrip workers all take in an argument called "targetMessagesPerSec". That argument works as an upper bound on the number of messages that can be consumed/produced per second in that worker.

It is useful to support infinite messages per second. Currently, if the `targetMessagesPerSec` field is not present in the request, the RoundTripWorker will raise an exception, whereas the ConsumeBench and ProduceBench workers will work as if they had `targetMessagesPerSec=10`.

I propose we allow for unbounded `targetMessagesPerSec` if the field is not present.
Further, it would be very useful if some of these workers showed the `messagesPerSecond` they have been producing/consuming at.
Even now, giving the worker a `targetMessagesPerSec` does not guarantee that the worker will reach the needed `targetMessagesPerSec`. There is no easy way of knowing how the worker performed - you have to subtract the status fields `startedMs` and `doneMs` to get the total duration of the task, convert to seconds and then divide that by the `maxMessages` field.

Issue Links

• links to: GitHub Pull Request #8434