Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9854

Collect metrics for index merges and index store IO

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 6.4, 7.0
    • Fix Version/s: 6.4, 7.0
    • Component/s: metrics
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None

      Description

      Using API for metrics management developed in SOLR-4735 we should also start collecting metrics for major aspects of IndexWriter operation, such as read / write IO rates, number of minor and major merges and IO during these operations, etc.

      This will provide a better insight into resource consumption and load at the IO level.

      1. SOLR-9854.patch
        54 kB
        Andrzej Bialecki
      2. SOLR-9854.patch
        37 kB
        Andrzej Bialecki

        Issue Links

          Activity

          Hide
          ab Andrzej Bialecki added a comment - - edited

          Patch relative to master. See the details on branch jira/solr-9854, here's a short log of key commits from this branch:

          commit 6ceb05074b3f6303e660613e277c3e26472ec627
          Author: Andrzej Bialecki <ab@apache.org>
          Date:   Fri Dec 23 21:30:15 2016 +0100
          
              SOLR-9854 Move metrics collection to SolrIndexWriter to get more meaningful
              timings of merges. Other changes:
              * simplify metric names in MetricUtils. Don't scale units for non-timer histograms
                as they may represent other units (eg. bytes)
              * add ability to configure metric details and parameters (currently the threshold
                between minor and major merges).
          ...
          commit ce8ab8c6359753623ae7a96204badd16193efc09
          Author: Andrzej Bialecki <ab@apache.org>
          Date:   Thu Dec 15 13:33:28 2016 +0100
          
              SOLR-9854 Initial version. This adds metrics to Directory IO and also
              monitors minor / major merge operations.
          
          Show
          ab Andrzej Bialecki added a comment - - edited Patch relative to master. See the details on branch jira/solr-9854 , here's a short log of key commits from this branch: commit 6ceb05074b3f6303e660613e277c3e26472ec627 Author: Andrzej Bialecki <ab@apache.org> Date: Fri Dec 23 21:30:15 2016 +0100 SOLR-9854 Move metrics collection to SolrIndexWriter to get more meaningful timings of merges. Other changes: * simplify metric names in MetricUtils. Don't scale units for non-timer histograms as they may represent other units (eg. bytes) * add ability to configure metric details and parameters (currently the threshold between minor and major merges). ... commit ce8ab8c6359753623ae7a96204badd16193efc09 Author: Andrzej Bialecki <ab@apache.org> Date: Thu Dec 15 13:33:28 2016 +0100 SOLR-9854 Initial version. This adds metrics to Directory IO and also monitors minor / major merge operations.
          Hide
          ab Andrzej Bialecki added a comment -

          Current patch. I think this is ready.

          Show
          ab Andrzej Bialecki added a comment - Current patch. I think this is ready.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 48ca9fc3f4f8d95293cee7bb59eff61247ede181 in lucene-solr's branch refs/heads/master from Andrzej Bialecki
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=48ca9fc ]

          SOLR-9854: Collect metrics for index merges and index store IO. (squashed)

          Show
          jira-bot ASF subversion and git services added a comment - Commit 48ca9fc3f4f8d95293cee7bb59eff61247ede181 in lucene-solr's branch refs/heads/master from Andrzej Bialecki [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=48ca9fc ] SOLR-9854 : Collect metrics for index merges and index store IO. (squashed)
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit c40cd2df49c80aee1ab2b6fea634191edc8b944f in lucene-solr's branch refs/heads/branch_6x from Andrzej Bialecki
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c40cd2d ]

          SOLR-9854: Collect metrics for index merges and index store IO (squashed).

          Show
          jira-bot ASF subversion and git services added a comment - Commit c40cd2df49c80aee1ab2b6fea634191edc8b944f in lucene-solr's branch refs/heads/branch_6x from Andrzej Bialecki [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c40cd2d ] SOLR-9854 : Collect metrics for index merges and index store IO (squashed).
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit b9827bcba9ee4e1b0dea8a18c6efe4322a400b09 in lucene-solr's branch refs/heads/master from Andrzej Bialecki
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b9827bc ]

          SOLR-9854 Relax test assertions.

          Show
          jira-bot ASF subversion and git services added a comment - Commit b9827bcba9ee4e1b0dea8a18c6efe4322a400b09 in lucene-solr's branch refs/heads/master from Andrzej Bialecki [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b9827bc ] SOLR-9854 Relax test assertions.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 03a41258b13075ff2fd305f538c82241503d7821 in lucene-solr's branch refs/heads/branch_6x from Andrzej Bialecki
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=03a4125 ]

          SOLR-9854 Relax test assertions.

          Show
          jira-bot ASF subversion and git services added a comment - Commit 03a41258b13075ff2fd305f538c82241503d7821 in lucene-solr's branch refs/heads/branch_6x from Andrzej Bialecki [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=03a4125 ] SOLR-9854 Relax test assertions.

            People

            • Assignee:
              ab Andrzej Bialecki
              Reporter:
              ab Andrzej Bialecki
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development