Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9880

Add Ganglia, Graphite and SLF4J metrics reporters

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.4, 7.0
    • Component/s: metrics
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None

      Description

      Originally SOLR-4735 provided implementations for these reporters (wrappers for Dropwizard components to use SolrMetricReporter API).

      However, this functionality has been split into its own issue due to the additional transitive dependencies that these reporters bring:

      • Ganglia:
        • metrics-ganglia, ASL, 3kB
        • gmetric4j (Ganglia RPC implementation), BSD, 29kB
      • Graphite
        • metrics-graphite, ASL, 10kB
        • amqp-client (RabbitMQ Java client, marked optional in pom?), ASL/MIT/GPL2, 190kB

      IMHO these are not very large dependencies, and given the useful functionality they provide it's worth adding them.

      1. SOLR-9880.patch
        33 kB
        Andrzej Bialecki

        Activity

        Hide
        yriveiro Yago Riveiro added a comment -

        Any chance to add too the metrics-zabbix report?

        https://github.com/hengyunabc/metrics-zabbix

        Show
        yriveiro Yago Riveiro added a comment - Any chance to add too the metrics-zabbix report? https://github.com/hengyunabc/metrics-zabbix
        Hide
        ab Andrzej Bialecki added a comment -

        This looks like a very young project, and it brings Alibaba's fastjson dependency at 290kB. I'd rather tackle this as a separate issue.

        Show
        ab Andrzej Bialecki added a comment - This looks like a very young project, and it brings Alibaba's fastjson dependency at 290kB. I'd rather tackle this as a separate issue.
        Hide
        otis Otis Gospodnetic added a comment -

        May I recommend a different approach? With the current approach you'll always have somebody come in and ask for additional reporter and typically that won't be very high on Solr devs' interest, plus it will require more work, additional dependencies, etc. Moreover, if you do this then you have to think about things like destination not being available, about possible on-disk buffering so data is not lost, about ensuring the buffered data is purged if there is too much of it, and so on. Solr doesn't want to be in data shipper business. My suggestion, based on working with monitoring and logging for the last N years - just log metrics to a file. There are a number of modern tools that know how to tail files, parse their content, ship it somewhere, have buffering, have multiple outputs, and so on. Just make sure data is nicely structured to make parsing easy, and done in a way that when you add more metrics you can do it in a backwards-compatible way.

        Show
        otis Otis Gospodnetic added a comment - May I recommend a different approach? With the current approach you'll always have somebody come in and ask for additional reporter and typically that won't be very high on Solr devs' interest, plus it will require more work, additional dependencies, etc. Moreover, if you do this then you have to think about things like destination not being available, about possible on-disk buffering so data is not lost, about ensuring the buffered data is purged if there is too much of it, and so on. Solr doesn't want to be in data shipper business. My suggestion, based on working with monitoring and logging for the last N years - just log metrics to a file. There are a number of modern tools that know how to tail files, parse their content, ship it somewhere, have buffering, have multiple outputs, and so on. Just make sure data is nicely structured to make parsing easy, and done in a way that when you add more metrics you can do it in a backwards-compatible way.
        Hide
        ab Andrzej Bialecki added a comment -

        I understand your concern. However, these two reporters seemed like a natural addition for several reasons:

        • they are already a part of the Metrics distribution,
        • they are maintained in the upstream package.
        • they integrate with popular systems,
        • in particular, Graphite protocol is so dead simple that it could be re-used by other systems, too - it just sends plain-text lines to a socket, with the name of the metric, its value and timestamp separated by space:
          myPrefix.solr.node.cores.lazy 0 1482394715
          myPrefix.solr.node.cores.loaded 1 1482394715
          myPrefix.solr.node.cores.unloaded 0 1482394715
          

        But you raise a good point - we should provide a file-based reporter too, as the lowest common denominator reporting mechanism... In fact, one is already included in the Metrics distribution - the Slf4jReporter - and it's trivial to add a wrapper. This reporter could use a Logger per metrics group (configurable), so it should be easy to tweak the logging configuration so that its output goes to separate file(s) with rolling.

        Show
        ab Andrzej Bialecki added a comment - I understand your concern. However, these two reporters seemed like a natural addition for several reasons: they are already a part of the Metrics distribution, they are maintained in the upstream package. they integrate with popular systems, in particular, Graphite protocol is so dead simple that it could be re-used by other systems, too - it just sends plain-text lines to a socket, with the name of the metric, its value and timestamp separated by space: myPrefix.solr.node.cores.lazy 0 1482394715 myPrefix.solr.node.cores.loaded 1 1482394715 myPrefix.solr.node.cores.unloaded 0 1482394715 But you raise a good point - we should provide a file-based reporter too, as the lowest common denominator reporting mechanism... In fact, one is already included in the Metrics distribution - the Slf4jReporter - and it's trivial to add a wrapper. This reporter could use a Logger per metrics group (configurable), so it should be easy to tweak the logging configuration so that its output goes to separate file(s) with rolling.
        Hide
        yriveiro Yago Riveiro added a comment -

        With a file-based report I can do a wrapper to read the file, that is ok and for integrations I think it's the better way, plain-text is unix friendly

        +1 to have a file-based report.

        Show
        yriveiro Yago Riveiro added a comment - With a file-based report I can do a wrapper to read the file, that is ok and for integrations I think it's the better way, plain-text is unix friendly +1 to have a file-based report.
        Hide
        ab Andrzej Bialecki added a comment - - edited

        Here's a patch that adds three new reporters for Ganglia, Graphite and Slf4j.

        SolrSlf4jReporter is file-based, ie. you can configure its logger name and redirect these logs to separate file(s). Logger names follow the metric group names by default, so you can have different files for "solr.jvm", "solr.core", "solr.node" etc - or you can override this and put all groups into a single logger.

        Show
        ab Andrzej Bialecki added a comment - - edited Here's a patch that adds three new reporters for Ganglia, Graphite and Slf4j. SolrSlf4jReporter is file-based, ie. you can configure its logger name and redirect these logs to separate file(s). Logger names follow the metric group names by default, so you can have different files for "solr.jvm", "solr.core", "solr.node" etc - or you can override this and put all groups into a single logger.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 9d606d8ff058b5a66b53b9255317c139c53effc7 in lucene-solr's branch refs/heads/master from Andrzej Bialecki
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9d606d8 ]

        SOLR-9880: Add Ganglia, Graphite and SLF4J metrics reporters.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 9d606d8ff058b5a66b53b9255317c139c53effc7 in lucene-solr's branch refs/heads/master from Andrzej Bialecki [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9d606d8 ] SOLR-9880 : Add Ganglia, Graphite and SLF4J metrics reporters.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit bc55d8bc427bcd3ebc6fafe8af9fc9d625fd104c in lucene-solr's branch refs/heads/branch_6x from Andrzej Bialecki
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=bc55d8b ]

        SOLR-9880: Add Ganglia, Graphite and SLF4J metrics reporters.

        Show
        jira-bot ASF subversion and git services added a comment - Commit bc55d8bc427bcd3ebc6fafe8af9fc9d625fd104c in lucene-solr's branch refs/heads/branch_6x from Andrzej Bialecki [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=bc55d8b ] SOLR-9880 : Add Ganglia, Graphite and SLF4J metrics reporters.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit fb2800b1497a67493c7f8944bda22f590bb9cc6b in lucene-solr's branch refs/heads/master from Andrzej Bialecki
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=fb2800b ]

        SOLR-9880 Fix concurrency bugs in tests.

        Show
        jira-bot ASF subversion and git services added a comment - Commit fb2800b1497a67493c7f8944bda22f590bb9cc6b in lucene-solr's branch refs/heads/master from Andrzej Bialecki [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=fb2800b ] SOLR-9880 Fix concurrency bugs in tests.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 250036e29d21d1e86a8484dc84be51111caa40b8 in lucene-solr's branch refs/heads/branch_6x from Andrzej Bialecki
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=250036e ]

        SOLR-9880 Fix concurrency bugs in tests.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 250036e29d21d1e86a8484dc84be51111caa40b8 in lucene-solr's branch refs/heads/branch_6x from Andrzej Bialecki [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=250036e ] SOLR-9880 Fix concurrency bugs in tests.

          People

          • Assignee:
            ab Andrzej Bialecki
            Reporter:
            ab Andrzej Bialecki
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development