Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-10247

Support non-numeric metrics and a "compact" format of /admin/metrics

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.5, 7.0
    • Component/s: metrics
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None

      Description

      Metrics API currently supports only numeric values. However, it's useful also to report non-numeric values such as eg. version, disk type, component state, some system properties, etc.

      Codahale Gauge<T> metric type can be used for this purpose, and convenience methods can be added to SolrMetricManager to make it easier to use.

      1. compactFormat.png
        241 kB
        Andrzej Bialecki
      2. currentFormat.png
        143 kB
        Andrzej Bialecki
      3. SOLR-10247.patch
        43 kB
        Andrzej Bialecki
      4. SOLR-10247.patch
        42 kB
        Andrzej Bialecki

        Activity

        Hide
        ab Andrzej Bialecki added a comment - - edited

        Initial patch. Some classes have been converted to use the new style for Gauge registration eg. SolrIndexWriter. See SolrCore.initializeMetrics for an example of non-numeric registrations.

        Also, I added the ability to produce a much more compact NamedList format in MetricUtils, which is used in the output from /admin/metrics. We should consider using it by default, although it's not back-compatible with the format in 6x.

        Show
        ab Andrzej Bialecki added a comment - - edited Initial patch. Some classes have been converted to use the new style for Gauge registration eg. SolrIndexWriter . See SolrCore.initializeMetrics for an example of non-numeric registrations. Also, I added the ability to produce a much more compact NamedList format in MetricUtils , which is used in the output from /admin/metrics . We should consider using it by default, although it's not back-compatible with the format in 6x.
        Hide
        shalinmangar Shalin Shekhar Mangar added a comment -
        1. The patch removes the following gauges from UpdateLog: replayLogsCountGauge, replayBytesGauge, stateGauge – I think this is a mistake?
        2. nit - DirectUpdateHandlerTest has an unused import which will cause precommit to fail
        3. In IndexWriter's constructor, while registering the gauge () -> runningMajorMerges – shouldn't that be () -> runningMajorMerges.get()? and so on for all others
        4. I like the compact output – we can add a version parameter to the MetricsHandler, say version=2, that outputs the compact format – this can become the default in 7.0
        Show
        shalinmangar Shalin Shekhar Mangar added a comment - The patch removes the following gauges from UpdateLog: replayLogsCountGauge, replayBytesGauge, stateGauge – I think this is a mistake? nit - DirectUpdateHandlerTest has an unused import which will cause precommit to fail In IndexWriter's constructor, while registering the gauge () -> runningMajorMerges – shouldn't that be () -> runningMajorMerges.get() ? and so on for all others I like the compact output – we can add a version parameter to the MetricsHandler, say version=2, that outputs the compact format – this can become the default in 7.0
        Hide
        ab Andrzej Bialecki added a comment -

        1. It doesn't, they are now registered as inline lambdas.
        3. Yes, I'll fix this.
        4. Good idea, I'll add this.

        Show
        ab Andrzej Bialecki added a comment - 1. It doesn't, they are now registered as inline lambdas. 3. Yes, I'll fix this. 4. Good idea, I'll add this.
        Hide
        ab Andrzej Bialecki added a comment -

        Updated patch.

        Show
        ab Andrzej Bialecki added a comment - Updated patch.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit cd4cc72b8ea27dba111c6bcd874bc1726b0f8e05 in lucene-solr's branch refs/heads/branch_6x from Andrzej Bialecki
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=cd4cc72 ]

        SOLR-10247: Support non-numeric metrics and a "compact" format of /admin/metrics.

        Show
        jira-bot ASF subversion and git services added a comment - Commit cd4cc72b8ea27dba111c6bcd874bc1726b0f8e05 in lucene-solr's branch refs/heads/branch_6x from Andrzej Bialecki [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=cd4cc72 ] SOLR-10247 : Support non-numeric metrics and a "compact" format of /admin/metrics.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 0e3b0b05fc1e1faf8943d8a55bdc62141c10583b in lucene-solr's branch refs/heads/branch_6x from Andrzej Bialecki
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=0e3b0b0 ]

        SOLR-10247: Fix a cherry-pick issue.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 0e3b0b05fc1e1faf8943d8a55bdc62141c10583b in lucene-solr's branch refs/heads/branch_6x from Andrzej Bialecki [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=0e3b0b0 ] SOLR-10247 : Fix a cherry-pick issue.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1750095048a5bee9c8f63d9bf6ad53d4a309aa60 in lucene-solr's branch refs/heads/master from Andrzej Bialecki
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1750095 ]

        SOLR-10247: Support non-numeric metrics and a "compact" format of /admin/metrics.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1750095048a5bee9c8f63d9bf6ad53d4a309aa60 in lucene-solr's branch refs/heads/master from Andrzej Bialecki [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1750095 ] SOLR-10247 : Support non-numeric metrics and a "compact" format of /admin/metrics.
        Hide
        otis Otis Gospodnetic added a comment -

        "compact" format of /admin/metrics

        Something like /cat.... in ES or something different? /cat in ES is handy...

        Show
        otis Otis Gospodnetic added a comment - "compact" format of /admin/metrics Something like /cat.... in ES or something different? /cat in ES is handy...
        Hide
        ab Andrzej Bialecki added a comment - - edited

        If I understand that functionality in ES (from casual reading of the docs) it's more like a different wt in Solr. The "compact" format provides just more terse output for existing writer types.

        It's possible that due to my lack of familiarity with ES I'm missing something that is worth further investigation - if so, let's do this in a separate Jira issue.

        Show
        ab Andrzej Bialecki added a comment - - edited If I understand that functionality in ES (from casual reading of the docs) it's more like a different wt in Solr. The "compact" format provides just more terse output for existing writer types. It's possible that due to my lack of familiarity with ES I'm missing something that is worth further investigation - if so, let's do this in a separate Jira issue.
        Hide
        otis Otis Gospodnetic added a comment -

        Short version - /cat provides table-like output - columns, with optional header, more or less verbose. Handy for piping into sort and friends that humans like to use, but also handy for agents because its output is simpler/cheaper to parse than JSON.

        Show
        otis Otis Gospodnetic added a comment - Short version - /cat provides table-like output - columns, with optional header, more or less verbose. Handy for piping into sort and friends that humans like to use, but also handy for agents because its output is simpler/cheaper to parse than JSON.
        Hide
        ab Andrzej Bialecki added a comment -

        Then yeah, I would argue that this is equivalent to a different response writer type, which seems like a different issue.

        Show
        ab Andrzej Bialecki added a comment - Then yeah, I would argue that this is equivalent to a different response writer type, which seems like a different issue.

          People

          • Assignee:
            ab Andrzej Bialecki
            Reporter:
            ab Andrzej Bialecki
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development