Solr
  1. Solr
  2. SOLR-1101

expose extra statistics like slow queries

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 4.9, 5.0
    • Component/s: web gui
    • Labels:
      None

      Description

      Solr Objects may want to expose extra information for diagnostics etc. These information can be large and we may not wish to show it in the stats page which can make it unwieldly. I propose an extension of the stats itself which can be shown separately.

      I wish to implement slow queries as a part of this. There is a new interface which can be implemented by components

      public interface StatsDetails {
      
      public interface StatsDetails {
      
        /**The details info is a Map of details name versus description. The details name must
         * return the details when the getDetails() method is invoked on the same object.
         * @return a map of details name vs the description. the description may be used in a link text
         */
        public Map<String ,String> getDetailsInfo();
      
        /**The details key should be known for this Object . 
         * This key will may be obj=tained by invoking getDetailsInfo() 
         * 
         * @param detailsCommand
         * @return
         */
        public NamedList getDetails(String detailsCommand);
      
      
      }
      

      The stats.jsp can be enhanced to show these extra links .

        Issue Links

          Activity

          Hide
          Uwe Schindler added a comment -

          Move issue to Solr 4.9.

          Show
          Uwe Schindler added a comment - Move issue to Solr 4.9.
          Hide
          Steve Rowe added a comment -

          Bulk move 4.4 issues to 4.5 and 5.0

          Show
          Steve Rowe added a comment - Bulk move 4.4 issues to 4.5 and 5.0
          Hide
          Hoss Man added a comment -

          Bulk of fixVersion=3.6 -> fixVersion=4.0 for issues that have no assignee and have not been updated recently.

          email notification suppressed to prevent mass-spam
          psuedo-unique token identifying these issues: hoss20120321nofix36

          Show
          Hoss Man added a comment - Bulk of fixVersion=3.6 -> fixVersion=4.0 for issues that have no assignee and have not been updated recently. email notification suppressed to prevent mass-spam psuedo-unique token identifying these issues: hoss20120321nofix36
          Hide
          Robert Muir added a comment -

          3.4 -> 3.5

          Show
          Robert Muir added a comment - 3.4 -> 3.5
          Hide
          Robert Muir added a comment -

          Bulk move 3.2 -> 3.3

          Show
          Robert Muir added a comment - Bulk move 3.2 -> 3.3
          Hide
          Hoss Man added a comment -

          Bulk updating 240 Solr issues to set the Fix Version to "next" per the process outlined in this email...

          http://mail-archives.apache.org/mod_mbox/lucene-dev/201005.mbox/%3Calpine.DEB.1.10.1005251052040.24672@radix.cryptio.net%3E

          Selection criteria was "Unresolved" with a Fix Version of 1.5, 1.6, 3.1, or 4.0. email notifications were suppressed.

          A unique token for finding these 240 issues in the future: hossversioncleanup20100527

          Show
          Hoss Man added a comment - Bulk updating 240 Solr issues to set the Fix Version to "next" per the process outlined in this email... http://mail-archives.apache.org/mod_mbox/lucene-dev/201005.mbox/%3Calpine.DEB.1.10.1005251052040.24672@radix.cryptio.net%3E Selection criteria was "Unresolved" with a Fix Version of 1.5, 1.6, 3.1, or 4.0. email notifications were suppressed. A unique token for finding these 240 issues in the future: hossversioncleanup20100527
          Hide
          Noble Paul added a comment - - edited

          Hi Hoss ,
          you are right. It has no means to drill down further

          How about this.

          
          public interface StatsDetails {
          
            /**The details info is a Map of details name versus description. The details name must
             * return the details when the getDetails() method is invoked on the same object.
             * @return a map of details name vs the description. the description may be used in a link text
             */
            public Map<String ,String> getDetailsInfo();
          
            /**The details key should be known for this Object . 
             * This key can be obtained by invoking getDetailsInfo() 
             * 
             * @param detailsCommand
             * @param params Any extra parameters which the command may understand. can be null also. 
             * @return
             */
            public NamedList getDetails(String detailsCommand, SolrParams params);
          
          }
          
          
          Show
          Noble Paul added a comment - - edited Hi Hoss , you are right. It has no means to drill down further How about this. public interface StatsDetails { /**The details info is a Map of details name versus description. The details name must * return the details when the getDetails() method is invoked on the same object. * @ return a map of details name vs the description. the description may be used in a link text */ public Map< String , String > getDetailsInfo(); /**The details key should be known for this Object . * This key can be obtained by invoking getDetailsInfo() * * @param detailsCommand * @param params Any extra parameters which the command may understand. can be null also. * @ return */ public NamedList getDetails( String detailsCommand, SolrParams params); }
          Hide
          Hoss Man added a comment -

          A simple string may not be quite sufficient

          sure ... no disagreement.

          my point is that instead of a marker interface that the components implement (which would only allow one level of drill down) the marker interface could be on new objects put into the NamedList returned by the existing getStatistics() method – when that interface is detected, show getLabel(), otherwise show toString().

          the extended display (extrastats.jsp if you want, or some new option on stats.jsp) would be able to getDetails() on the object ... and any of those details might themselves implement StatsDetails to allow further drilldown.

          Show
          Hoss Man added a comment - A simple string may not be quite sufficient sure ... no disagreement. my point is that instead of a marker interface that the components implement (which would only allow one level of drill down) the marker interface could be on new objects put into the NamedList returned by the existing getStatistics() method – when that interface is detected, show getLabel(), otherwise show toString(). the extended display (extrastats.jsp if you want, or some new option on stats.jsp) would be able to getDetails() on the object ... and any of those details might themselves implement StatsDetails to allow further drilldown.
          Hide
          Noble Paul added a comment -

          is that what you had in mind?

          Yes. The understanding is correct. Just that the detailed stats may be rendered by a separate jsp say extrastats.jsp?obj=foo&detail=name

          The XSL based rendering is a problem for the current stats.jsp . XSL is too cumbersome for any advanced UI

          The default stats.jsp can provide links with this information so that users can navigate from there and the current stats.jsp can be a entry point for further drill down. But it should also be possible to hit the extrastats.jsp directly which can just provide the links to all the objects with extra statistics

          Another thing to consider is that stats has always displayed the toString() of the values in the getStatistics() NamedList

          The concern here is that, the extra stats will be like mini apps which will give options to view and mine tons of data,
          eg:

          • contents of my queryResultCache
            • hits for some items etc
          • all the queries which took more than 100 ms
            • or the ones where this 'fq' is appplied
              *etc etc

          These things would need features like faceted drill down, pagination etc. A simple string may not be quite sufficient

          Show
          Noble Paul added a comment - is that what you had in mind? Yes. The understanding is correct. Just that the detailed stats may be rendered by a separate jsp say extrastats.jsp?obj=foo&detail=name The XSL based rendering is a problem for the current stats.jsp . XSL is too cumbersome for any advanced UI The default stats.jsp can provide links with this information so that users can navigate from there and the current stats.jsp can be a entry point for further drill down. But it should also be possible to hit the extrastats.jsp directly which can just provide the links to all the objects with extra statistics Another thing to consider is that stats has always displayed the toString() of the values in the getStatistics() NamedList The concern here is that, the extra stats will be like mini apps which will give options to view and mine tons of data, eg: contents of my queryResultCache hits for some items etc all the queries which took more than 100 ms or the ones where this 'fq' is appplied *etc etc These things would need features like faceted drill down, pagination etc. A simple string may not be quite sufficient
          Hide
          Hoss Man added a comment - - edited

          Noble: I presume by "There is a new interface which can be implemented by components" you're suggesting that this StatsDetails could be a marker interface implemented by things that already implement SolrInfoMBean, and if stats.jsp noticed something implementing this interface, then it would call getDetailsInfo() and for each name returned include a link to a future call of something like "stats.jsp?obj=foo&detail=name" which would call getDetails(name)

          is that what you had in mind?

          Another thing to consider is that stats has always displayed the toString() of the values in the getStatistics() NamedList ... perhaps a marker interface could be added to those objects, indicating when the toString behavior should be overridden and replaced with the ability to drill down...

          public interface DetailedStats {
             public String getSummary();
             public NamedList getDetails();
          }
          public class DetailedStatsNamedList extends NamedList implements DetailedStats {
             public DetailedStatsNamedList(String summary) { this.summary = summary; }
             public NamedList getDetails() return this;
             // ... hashCode & equals & toString
          }
          ...
          // stats.jsp would print getSummary() as a link to unwrapping getDetails()
          ...
          

          This would have the advantage of allowing deeper and deeper levels of details to be exposed if the stats.jsp code was written generally enough.

          Show
          Hoss Man added a comment - - edited Noble: I presume by "There is a new interface which can be implemented by components" you're suggesting that this StatsDetails could be a marker interface implemented by things that already implement SolrInfoMBean, and if stats.jsp noticed something implementing this interface, then it would call getDetailsInfo() and for each name returned include a link to a future call of something like "stats.jsp?obj=foo&detail=name" which would call getDetails(name) is that what you had in mind? Another thing to consider is that stats has always displayed the toString() of the values in the getStatistics() NamedList ... perhaps a marker interface could be added to those objects, indicating when the toString behavior should be overridden and replaced with the ability to drill down... public interface DetailedStats { public String getSummary(); public NamedList getDetails(); } public class DetailedStatsNamedList extends NamedList implements DetailedStats { public DetailedStatsNamedList( String summary) { this .summary = summary; } public NamedList getDetails() return this ; // ... hashCode & equals & toString } ... // stats.jsp would print getSummary() as a link to unwrapping getDetails() ... This would have the advantage of allowing deeper and deeper levels of details to be exposed if the stats.jsp code was written generally enough.

            People

            • Assignee:
              Unassigned
              Reporter:
              Noble Paul
            • Votes:
              6 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:

                Development