Kafka
  1. Kafka
  2. KAFKA-1190

create a draw performance graph script

    Details

    • Type: Improvement Improvement
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 0.8.2
    • Component/s: None
    • Labels:
      None

      Description

      This will be an R script to draw relevant graphs given a bunch of csv files from the above tools. The output of this script will be a bunch of png files that can be combined with some html to act as a perf report.

      Here are the graphs that would be good to see:

      • Latency histogram for producer
      • MB/sec and messages/sec produced
      • MB/sec and messages/sec consumed
      • Flush time
      • Errors (should not be any)
      • Consumer cache hit ratio (both the bytes and count, specifically 1
        #physical_reads / #requests and 1 - physical_bytes_read / bytes_read)
      • Write merge ratio (num_physical_writes/num_produce_requests and avg_request_size/avg_physical_write_size)
        CPU, network, io, etc

        Activity

        Hide
        Joe Stein added a comment -

        Created reviewboard https://reviews.apache.org/r/16714/
        against branch origin/trunk

        Show
        Joe Stein added a comment - Created reviewboard https://reviews.apache.org/r/16714/ against branch origin/trunk
        Hide
        Joel Koshy added a comment -

        This is useful but do we want to check-in very specific scripts as opposed to an external contrib location or even one of the wikis? csv is a general-enough raw format that people can read to generate graphs/reports using R/gnuplot/matlab/excel/whatever number crunchers they like.

        Show
        Joel Koshy added a comment - This is useful but do we want to check-in very specific scripts as opposed to an external contrib location or even one of the wikis? csv is a general-enough raw format that people can read to generate graphs/reports using R/gnuplot/matlab/excel/whatever number crunchers they like.
        Hide
        Neha Narkhede added a comment -

        Patch pending, moving out of 0.8.1

        Show
        Neha Narkhede added a comment - Patch pending, moving out of 0.8.1

          People

          • Assignee:
            Unassigned
            Reporter:
            Joe Stein
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Development