Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-7599

Trogdor - Allow configuration for not throttling Benchmark Workers and expose messages per second in task status

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      In Trogdor, the ConsumeBench, ProduceBench and RoundTrip workers all take in an argument called "targetMessagesPerSec". That argument works as an upper bound on the number of messages that can be consumed/produced per second in that worker.

      It is useful to support infinite messages per second. Currently, if the `targetMessagesPerSec` field is not present in the request, the RoundTripWorker will raise an exception, whereas the ConsumeBench and ProduceBench workers will work as if they had `targetMessagesPerSec=10`.

      I propose we allow for unbounded `targetMessagesPerSec` if the field is not present.
      Further, it would be very useful if some of these workers showed the `messagesPerSecond` they have been producing/consuming at.
      Even now, giving the worker a `targetMessagesPerSec` does not guarantee that the worker will reach the needed `targetMessagesPerSec`. There is no easy way of knowing how the worker performed - you have to subtract the status fields `startedMs` and `doneMs` to get the total duration of the task, convert to seconds and then divide that by the `maxMessages` field.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                enether Stanislav Kozlovski
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: