Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-2643

Add tools to do basic analysis on the metrics logs

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.8.0
    • Fix Version/s: None
    • Component/s: None

      Description

      KUDU-2597 tracks adding a tool to parse the metrics logs. We should also add (probably as Python scripts) some tools for analyzing the metrics logs:

      • Finding tablets with unusual performance characteristics: longest apply|prepare|replicate|write|updateconsensus times
      • Finding servers with the most disk activity
      • Finding servers with slows scanners
      • Finding replicas that are largest on disk or in-memory
      • Characterizing workloads of tables and tablets (insert/upserts/deletes/updates/duplicate key inserts/pk lookups + times/op)
      • Compaction (average height, delta size, compaction times)
      • Log performance (append latency, sync latency, throughput)

      Some of this can be rules-based, i.e. if metric X is > constant A) and some should be pattern-based (most tablets' histograms look different than this tablet's).

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                wdberkeley William Berkeley
                Reporter:
                wdberkeley William Berkeley
              • Votes:
                1 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: