Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Fix Version/s: 3.2
    • Component/s: None
    • Labels:

      Description

      Currently we only give a global number on how many compactions are pending, we should group this on a per-table basis, example:

      $ nodetool compactionstats
      pending tasks:
      - keyspace1.standard1: 10
      - other_ks.table: 2
      
      1. 10718-3.x.txt
        7 kB
        Taiyuan Zhang

        Activity

        Hide
        krummas Marcus Eriksson added a comment -

        committed, thanks

        Show
        krummas Marcus Eriksson added a comment - committed, thanks
        Hide
        firstprayer Taiyuan Zhang added a comment -

        Sorry, wasn't aware of the dtest... will remember next time.

        Show
        firstprayer Taiyuan Zhang added a comment - Sorry, wasn't aware of the dtest... will remember next time.
        Hide
        krummas Marcus Eriksson added a comment -

        just running the tests - we parse the output of compactionstats in dtests so I had to change the output a bit, updated patch here: https://github.com/krummas/cassandra/commits/marcuse/10718 and test runs here:
        http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-10718-dtest/
        http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-10718-testall/

        Show
        krummas Marcus Eriksson added a comment - just running the tests - we parse the output of compactionstats in dtests so I had to change the output a bit, updated patch here: https://github.com/krummas/cassandra/commits/marcuse/10718 and test runs here: http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-10718-dtest/ http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-10718-testall/
        Hide
        firstprayer Taiyuan Zhang added a comment -

        Thanks

        Show
        firstprayer Taiyuan Zhang added a comment - Thanks
        Hide
        krummas Marcus Eriksson added a comment -

        ah, of course, will get it committed, thanks

        Show
        krummas Marcus Eriksson added a comment - ah, of course, will get it committed, thanks
        Hide
        firstprayer Taiyuan Zhang added a comment -

        tableNameToCountMap is a map inside resultMap. resultMap is returned whenever the user reads from pendingTasksByTableName.

        Show
        firstprayer Taiyuan Zhang added a comment - tableNameToCountMap is a map inside resultMap . resultMap is returned whenever the user reads from pendingTasksByTableName .
        Hide
        krummas Marcus Eriksson added a comment -

        Taiyuan Zhang thanks, looks like you missed actually adding up the currently ongoing compactions from tableNameToCountMap in pendingTasksByTableName

        Show
        krummas Marcus Eriksson added a comment - Taiyuan Zhang thanks, looks like you missed actually adding up the currently ongoing compactions from tableNameToCountMap in pendingTasksByTableName
        Hide
        firstprayer Taiyuan Zhang added a comment -

        I added a patch for this ticket. I followed @Marcus 's advice and implement a new jmx method that returns a nested map, grouping the number of compaction tasks by keyspace then by table name.

        Show
        firstprayer Taiyuan Zhang added a comment - I added a patch for this ticket. I followed @Marcus 's advice and implement a new jmx method that returns a nested map, grouping the number of compaction tasks by keyspace then by table name.
        Hide
        krummas Marcus Eriksson added a comment -

        Shall I change the getValue() method (and call hierarchy) to return a Map<KeyspaceName, Map<tablename, pendingTasksValue>>

        You should probably add a new jmx method for this - I bet many operators graph the current pending compactions jmx value and we don't want to break that.

        Show
        krummas Marcus Eriksson added a comment - Shall I change the getValue() method (and call hierarchy) to return a Map<KeyspaceName, Map<tablename, pendingTasksValue>> You should probably add a new jmx method for this - I bet many operators graph the current pending compactions jmx value and we don't want to break that.
        Hide
        tusharice Tushar Agrawal added a comment -

        Need some help and guidance:

        PendingTasks metrics comes from "pendingTasks" gauge in CompactionMetrics.java. In getValue() method it is currently looping through all keyspaces and tables to get the estimated remaining tasks.

        Shall I change the getValue() method (and call hierarchy) to return a Map<KeyspaceName, Map<tablename, pendingTasksValue>> ? How this would be displayed under JMX metrics?

        What could be the other approaches?

        Show
        tusharice Tushar Agrawal added a comment - Need some help and guidance: PendingTasks metrics comes from "pendingTasks" gauge in CompactionMetrics.java. In getValue() method it is currently looping through all keyspaces and tables to get the estimated remaining tasks. Shall I change the getValue() method (and call hierarchy) to return a Map<KeyspaceName, Map<tablename, pendingTasksValue>> ? How this would be displayed under JMX metrics? What could be the other approaches?
        Hide
        krummas Marcus Eriksson added a comment -

        Tushar Agrawal have a look at the current compactionstats nodetool command, it is in CompactionStats.java

        Show
        krummas Marcus Eriksson added a comment - Tushar Agrawal have a look at the current compactionstats nodetool command, it is in CompactionStats.java
        Hide
        tusharice Tushar Agrawal added a comment -

        Any pointers on where to look into code for this issue? I am interested in taking this one.

        Show
        tusharice Tushar Agrawal added a comment - Any pointers on where to look into code for this issue? I am interested in taking this one.

          People

          • Assignee:
            tusharice Tushar Agrawal
            Reporter:
            krummas Marcus Eriksson
            Reviewer:
            Marcus Eriksson
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development