Kafka
  1. Kafka
  2. KAFKA-149

Current perf directory has buggy perf tests

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7
    • Component/s: None
    • Labels:
      None

      Description

      The scripts in the current perf directory are buggy and not useful to run any reliable Kafka performance tests. The performance tools that work correctly are -

      ProducerPerformance.scala
      SimpleConsumerPerformance.scala
      ConsumerPerformance.scala

      Currently, the above are in the tools directory. Ideally, a Kafka performance suite should repackage these tools with some sample performance load and output data in csv format that can be graphed.

      I suggest deleting the perf directory and redoing this cleanly.

        Activity

        Hide
        Chris Burroughs added a comment -

        Thanks, that makes sense. Is there a ticket yet for the New and Improved perf scripts?

        Show
        Chris Burroughs added a comment - Thanks, that makes sense. Is there a ticket yet for the New and Improved perf scripts?
        Hide
        Neha Narkhede added a comment -

        The perf directory had quite a few problems

        1. Our perf tools which are tested well were not used.
        2. No clear separation between perf tools and definition of performance load
        3. A lot of custom code written to plot some custom graphs, that cannot be easily understood by existing and new developers
        4. The custom code was in Java, and was buggy (the perf numbers that it gave wasn't exactly indicative of real Kafka performance)

        Ideally, our thought is to improve and bundle up the perf tools that we have and write some scripts that clearly define the performance load used and purpose of the perf test.

        Show
        Neha Narkhede added a comment - The perf directory had quite a few problems 1. Our perf tools which are tested well were not used. 2. No clear separation between perf tools and definition of performance load 3. A lot of custom code written to plot some custom graphs, that cannot be easily understood by existing and new developers 4. The custom code was in Java, and was buggy (the perf numbers that it gave wasn't exactly indicative of real Kafka performance) Ideally, our thought is to improve and bundle up the perf tools that we have and write some scripts that clearly define the performance load used and purpose of the perf test.
        Hide
        Chris Burroughs added a comment -

        I'm not following what was wrong with the perf/ tools, or what changed to make them unreliable.

        Show
        Chris Burroughs added a comment - I'm not following what was wrong with the perf/ tools, or what changed to make them unreliable.
        Neha Narkhede made changes -
        Field Original Value New Value
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Jun Rao added a comment -

        +1

        Show
        Jun Rao added a comment - +1
        Neha Narkhede created issue -

          People

          • Assignee:
            Unassigned
            Reporter:
            Neha Narkhede
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development