Kafka
  1. Kafka
  2. KAFKA-1190

create a draw performance graph script

    Details

    • Type: Improvement Improvement
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: None
    • Labels:
      None

      Description

      This will be an R script to draw relevant graphs given a bunch of csv files from the above tools. The output of this script will be a bunch of png files that can be combined with some html to act as a perf report.

      Here are the graphs that would be good to see:

      • Latency histogram for producer
      • MB/sec and messages/sec produced
      • MB/sec and messages/sec consumed
      • Flush time
      • Errors (should not be any)
      • Consumer cache hit ratio (both the bytes and count, specifically 1
        #physical_reads / #requests and 1 - physical_bytes_read / bytes_read)
      • Write merge ratio (num_physical_writes/num_produce_requests and avg_request_size/avg_physical_write_size)
        CPU, network, io, etc

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        15d 2h 40m 1 Joe Stein 08/Jan/14 00:42
        Guozhang Wang made changes -
        Fix Version/s 0.9.0 [ 12323928 ]
        Fix Version/s 0.8.2 [ 12326167 ]
        Hide
        Guozhang Wang added a comment -

        Moving out of 0.8.2 (again...)

        Show
        Guozhang Wang added a comment - Moving out of 0.8.2 (again...)
        Hide
        Neha Narkhede added a comment -

        Patch pending, moving out of 0.8.1

        Show
        Neha Narkhede added a comment - Patch pending, moving out of 0.8.1
        Neha Narkhede made changes -
        Fix Version/s 0.8.2 [ 12326167 ]
        Fix Version/s 0.8.1 [ 12322960 ]
        Hide
        Joel Koshy added a comment -

        This is useful but do we want to check-in very specific scripts as opposed to an external contrib location or even one of the wikis? csv is a general-enough raw format that people can read to generate graphs/reports using R/gnuplot/matlab/excel/whatever number crunchers they like.

        Show
        Joel Koshy added a comment - This is useful but do we want to check-in very specific scripts as opposed to an external contrib location or even one of the wikis? csv is a general-enough raw format that people can read to generate graphs/reports using R/gnuplot/matlab/excel/whatever number crunchers they like.
        Joe Stein made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Joe Stein added a comment -

        Created reviewboard https://reviews.apache.org/r/16714/
        against branch origin/trunk

        Show
        Joe Stein added a comment - Created reviewboard https://reviews.apache.org/r/16714/ against branch origin/trunk
        Joe Stein made changes -
        Attachment KAFKA-1190.patch [ 12621894 ]
        Joe Stein made changes -
        Field Original Value New Value
        Description This will be an R script to draw relevant graphs given a bunch of csv files from the above tools. The output of this script will be a bunch of png files that can be combined with some html to act as a perf report. This will be an R script to draw relevant graphs given a bunch of csv files from the above tools. The output of this script will be a bunch of png files that can be combined with some html to act as a perf report.

        Here are the graphs that would be good to see:
        * Latency histogram for producer
        * MB/sec and messages/sec produced
        * MB/sec and messages/sec consumed
        * Flush time
        * Errors (should not be any)
        * Consumer cache hit ratio (both the bytes and count, specifically 1
          #physical_reads / #requests and 1 - physical_bytes_read / bytes_read)
        * Write merge ratio (num_physical_writes/num_produce_requests and avg_request_size/avg_physical_write_size)
        CPU, network, io, etc
        Joe Stein created issue -

          People

          • Assignee:
            Unassigned
            Reporter:
            Joe Stein
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:

              Development