Solr
  1. Solr
  2. SOLR-6752

Buffer Cache allocate/lost metrics should be exposed

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.0, 6.0
    • Component/s: None
    • Labels:

      Description

      Currently, o.a.s.store.blockcache.Metrics has fields for tracking buffer allocations and losses, but they are never updated nor exposed to a receiving metrics system. We should do both.

      1. SOLR-6752.patch
        6 kB
        Mike Drob
      2. SOLR-6752.patch
        16 kB
        Mike Drob

        Issue Links

          Activity

          Hide
          Mike Drob added a comment -

          Patch applies to current trunk, ant clean test passed for me with no issues.

          Show
          Mike Drob added a comment - Patch applies to current trunk, ant clean test passed for me with no issues.
          Hide
          Mark Miller added a comment -

          SolrSOLR-6752 Buffer Cache allocate/lost should be exposed through JMX

          I'm a little confused by the title - isn't this simply exposing hdfds block cache metrics via JMX? Why anything specific about allocate/lost? I see all sorts of stats in the getStatistics call.

          We also probably want to try and align some of the stat key names with other cache objects in Solr: the query cache, filter cache, etc.

          Also, I don't believe these will get registered with the jmx server. I think only the top level class for a plugin is by default - eg the HdfsDirectoryFactory itself.

          Show
          Mark Miller added a comment - SolrSOLR-6752 Buffer Cache allocate/lost should be exposed through JMX I'm a little confused by the title - isn't this simply exposing hdfds block cache metrics via JMX? Why anything specific about allocate/lost? I see all sorts of stats in the getStatistics call. We also probably want to try and align some of the stat key names with other cache objects in Solr: the query cache, filter cache, etc. Also, I don't believe these will get registered with the jmx server. I think only the top level class for a plugin is by default - eg the HdfsDirectoryFactory itself.
          Hide
          Mike Drob added a comment -

          I'm a little confused by the title - isn't this simply exposing hdfds block cache metrics via JMX? Why anything specific about allocate/lost? I see all sorts of stats in the getStatistics call.

          The buffer allocatione/lost metrics are currently not exposed at all. When moving everything from metricsRecord.setMetric to stats.add in the metrics method, these lines are completely new, instead of being converted over.

          +      stats.add("buffercache.allocations", getPerSecond(shardBuffercacheAllocate.getAndSet(0), seconds));
          +      stats.add("buffercache.lost", getPerSecond(shardBuffercacheLost.getAndSet(0), seconds));
          

          We also probably want to try and align some of the stat key names with other cache objects in Solr: the query cache, filter cache, etc.

          This makes sense, I'll look into it.

          Also, I don't believe these will get registered with the jmx server. I think only the top level class for a plugin is by default - eg the HdfsDirectoryFactory itself.

          Not sure I understand this. Where should I look to make sure this is getting registered?

          Show
          Mike Drob added a comment - I'm a little confused by the title - isn't this simply exposing hdfds block cache metrics via JMX? Why anything specific about allocate/lost? I see all sorts of stats in the getStatistics call. The buffer allocatione/lost metrics are currently not exposed at all. When moving everything from metricsRecord.setMetric to stats.add in the metrics method, these lines are completely new, instead of being converted over. + stats.add("buffercache.allocations", getPerSecond(shardBuffercacheAllocate.getAndSet(0), seconds)); + stats.add("buffercache.lost", getPerSecond(shardBuffercacheLost.getAndSet(0), seconds)); We also probably want to try and align some of the stat key names with other cache objects in Solr: the query cache, filter cache, etc. This makes sense, I'll look into it. Also, I don't believe these will get registered with the jmx server. I think only the top level class for a plugin is by default - eg the HdfsDirectoryFactory itself. Not sure I understand this. Where should I look to make sure this is getting registered?
          Hide
          Mike Drob added a comment -

          I will split this issue into this and SOLR-6766 to reduce confusion.

          Show
          Mike Drob added a comment - I will split this issue into this and SOLR-6766 to reduce confusion.
          Hide
          Mike Drob added a comment -

          A patch for just tracking buffer cache allocate/lost metrics.

          Show
          Mike Drob added a comment - A patch for just tracking buffer cache allocate/lost metrics.
          Hide
          Mark Miller added a comment -

          Where should I look to make sure this is getting registered?

          I fired up Solr on HDFS with JMX enabled and took a look at the exported mbeans with JConsole. I did not see anything for the block cache.

          I'd look at how SolrResourceLoader adds the plugins that it loads to the JmxMonitoredMap.

          Show
          Mark Miller added a comment - Where should I look to make sure this is getting registered? I fired up Solr on HDFS with JMX enabled and took a look at the exported mbeans with JConsole. I did not see anything for the block cache. I'd look at how SolrResourceLoader adds the plugins that it loads to the JmxMonitoredMap.
          Hide
          Mike Drob added a comment -

          Any thoughts on the latest version?

          Show
          Mike Drob added a comment - Any thoughts on the latest version?
          Hide
          ASF subversion and git services added a comment -

          Commit 1642689 from Mark Miller in branch 'dev/trunk'
          [ https://svn.apache.org/r1642689 ]

          SOLR-6752: Buffer Cache allocate/lost metrics should be exposed.

          Show
          ASF subversion and git services added a comment - Commit 1642689 from Mark Miller in branch 'dev/trunk' [ https://svn.apache.org/r1642689 ] SOLR-6752 : Buffer Cache allocate/lost metrics should be exposed.
          Hide
          ASF subversion and git services added a comment -

          Commit 1642691 from Mark Miller in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1642691 ]

          SOLR-6752: Buffer Cache allocate/lost metrics should be exposed.

          Show
          ASF subversion and git services added a comment - Commit 1642691 from Mark Miller in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1642691 ] SOLR-6752 : Buffer Cache allocate/lost metrics should be exposed.
          Hide
          Mark Miller added a comment -

          Thanks Mike!

          Show
          Mark Miller added a comment - Thanks Mike!
          Hide
          Anshum Gupta added a comment -

          Bulk close after 5.0 release.

          Show
          Anshum Gupta added a comment - Bulk close after 5.0 release.

            People

            • Assignee:
              Mark Miller
              Reporter:
              Mike Drob
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development