Solr
  1. Solr
  2. SOLR-1750

SolrInfoMBeanHandler - replacement for stats.jsp and registry.jsp

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.5, 3.1, 4.0-ALPHA
    • Component/s: web gui
    • Labels:
      None

      Description

      stats.jsp is cool and all, but suffers from escaping issues, and also is not accessible from SolrJ or other standard Solr APIs.

      Here's a request handler that emits everything stats.jsp does.

      For now, it needs to be registered in solrconfig.xml like this:

          <requestHandler name="/admin/stats" class="solr.SystemStatsRequestHandler" />
      

      But will register this in AdminHandlers automatically before committing.

      1. SystemStatsRequestHandler.java
        2 kB
        Erik Hatcher
      2. SystemStatsRequestHandler.java
        4 kB
        Hoss Man
      3. SystemStatsRequestHandler.java
        4 kB
        Hoss Man
      4. SOLR-1750-followup.patch
        10 kB
        Hoss Man

        Activity

        Hide
        Erik Hatcher added a comment -

        I'll commit this in the near future.

        Any thoughts on the naming of this beast? "stats" is a bit overloaded (StatsComponent). as is "system" (SystemInfoHandler).

        Show
        Erik Hatcher added a comment - I'll commit this in the near future. Any thoughts on the naming of this beast? "stats" is a bit overloaded (StatsComponent). as is "system" (SystemInfoHandler).
        Hide
        Erik Hatcher added a comment -

        Also, food for thought, when (hopefully not if) the VelocityResponseWriter is moved into core, we can deprecate stats.jsp and skin the output of this request handler for a similar pleasant view like stats.jsp+client-side xsl does now.

        Show
        Erik Hatcher added a comment - Also, food for thought, when (hopefully not if) the VelocityResponseWriter is moved into core, we can deprecate stats.jsp and skin the output of this request handler for a similar pleasant view like stats.jsp+client-side xsl does now.
        Hide
        Steve Rowe added a comment -

        Any thoughts on the naming of this beast?

        How about SysInfoRequestHandler - bonus: SIRH evokes RFK's assassin

        Show
        Steve Rowe added a comment - Any thoughts on the naming of this beast? How about SysInfoRequestHandler - bonus: SIRH evokes RFK's assassin
        Hide
        Steve Rowe added a comment -

        "stats" is a bit overloaded (StatsComponent). as is "system" (SystemInfoHandler).

        I swear when I read this, before I suggested SIRH, you had written "SystemStatsHandler" instead of "SystemInfoHandler". Not sure how you changed it without a red "edited" annotation in the header for your comment.... Et tu, Atlassian?

        Anyway, pathological paranoia aside, SIRH is too close to SystemInfoHandler - I hereby begin the process of formally withdrawing it from consideration. Ok, done.

        stats.xsl creates a title prefix "Solr Statistics" - how about SolrStatsRequestHandler?

        Show
        Steve Rowe added a comment - "stats" is a bit overloaded (StatsComponent). as is "system" (SystemInfoHandler). I swear when I read this, before I suggested SIRH, you had written "SystemStatsHandler" instead of "SystemInfoHandler". Not sure how you changed it without a red "edited" annotation in the header for your comment.... Et tu, Atlassian? Anyway, pathological paranoia aside, SIRH is too close to SystemInfoHandler - I hereby begin the process of formally withdrawing it from consideration. Ok, done. stats.xsl creates a title prefix "Solr Statistics" - how about SolrStatsRequestHandler?
        Hide
        Simon Rosenthal added a comment -

        +1 on SolrStatsRequestHandler

        You might want to consider either omitting or making optional the Lucene Fieldcache stats; they can often be very slow to be generated ( see http://www.lucidimagination.com/search/document/5ba908577d2e4c25/stats_page_slow_in_latest_nightly#2f40166c25f9bfa0 ). One use case for this request handler that I can see is high frequency (every few seconds) monitoring as part of performance testing, for which a fast response is pretty mandatory.

        Show
        Simon Rosenthal added a comment - +1 on SolrStatsRequestHandler You might want to consider either omitting or making optional the Lucene Fieldcache stats; they can often be very slow to be generated ( see http://www.lucidimagination.com/search/document/5ba908577d2e4c25/stats_page_slow_in_latest_nightly#2f40166c25f9bfa0 ). One use case for this request handler that I can see is high frequency (every few seconds) monitoring as part of performance testing, for which a fast response is pretty mandatory.
        Hide
        Hoss Man added a comment -

        Any thoughts on the naming of this beast?

        SystemInfoHandler sounds good.

        This would probably also be a good time to retire "registry.jsp" ... all we need to do is add a few more pieces of "system info" to this handler (and add some param options to disable the "stats" part of the output)

        Also, food for thought, when (hopefully not if) the VelocityResponseWriter is moved into core, we can deprecate stats.jsp and skin the output of this request handler for a similar pleasant view like stats.jsp+client-side xsl does now.

        Even if/when VelocityResponseWRiter is in the core, i'd still rather just rely on client side XSLT for this to reduce the number of things that could potentially get missconfigured and then confuse people why the page doesn't look right ... the XmlResponseWRriter has always supported a "stylesheet" param that (while not generally useful to most people) let's you easily reference any style sheet that can be served out of the admin directory ... all we really need is an updatd .xsl file to translate the standard XML format into the old style stats view.

        Show
        Hoss Man added a comment - Any thoughts on the naming of this beast? SystemInfoHandler sounds good. This would probably also be a good time to retire "registry.jsp" ... all we need to do is add a few more pieces of "system info" to this handler (and add some param options to disable the "stats" part of the output) Also, food for thought, when (hopefully not if) the VelocityResponseWriter is moved into core, we can deprecate stats.jsp and skin the output of this request handler for a similar pleasant view like stats.jsp+client-side xsl does now. Even if/when VelocityResponseWRiter is in the core, i'd still rather just rely on client side XSLT for this to reduce the number of things that could potentially get missconfigured and then confuse people why the page doesn't look right ... the XmlResponseWRriter has always supported a "stylesheet" param that (while not generally useful to most people) let's you easily reference any style sheet that can be served out of the admin directory ... all we really need is an updatd .xsl file to translate the standard XML format into the old style stats view.
        Hide
        Hoss Man added a comment -

        Some updates to Erik's previous version...

        1. adds everything from registry.jsp
          • lucene/solr version info
          • source/docs info for each object
        2. forcibly disable HTTP Caching
        3. adds params to control which objects are listed
          • (multivalued) "cat" param restricts category names (default is all)
          • (multivalued) "key" param restricts object keys (default is all)
        4. adds (boolean) "stats" param to control if stats are outputed for each object
          • per-field style override can be used to override per object key
        5. refactored the old nested looping that stast.jsp did over every object and every category into a single pass
        6. switch all HashMaps to NamedLists or SimpleOrderedMaps to preserve predictable ordering

        Examples...

        • ?cat=CACHE
          • return info about caches, but nothing else (stats disabled by default)
        • ?stats=true&cat=CACHE
          • return info and stats about caches, but nothing else
        • ?stats=true&f.fieldCache.stats=false
          • Info about everything, stats for everything except fieldCache
        • ?key=fieldCache&stats=true
          • Return info and stats for fieldCache, but nothing else

        I left the class name alone, but i vote for "SystemInfoRequestHandler" with a default registration of "/admin/info"

        Show
        Hoss Man added a comment - Some updates to Erik's previous version... adds everything from registry.jsp lucene/solr version info source/docs info for each object forcibly disable HTTP Caching adds params to control which objects are listed (multivalued) "cat" param restricts category names (default is all) (multivalued) "key" param restricts object keys (default is all) adds (boolean) "stats" param to control if stats are outputed for each object per-field style override can be used to override per object key refactored the old nested looping that stast.jsp did over every object and every category into a single pass switch all HashMaps to NamedLists or SimpleOrderedMaps to preserve predictable ordering Examples... ?cat=CACHE return info about caches, but nothing else (stats disabled by default) ?stats=true&cat=CACHE return info and stats about caches, but nothing else ?stats=true&f.fieldCache.stats=false Info about everything, stats for everything except fieldCache ?key=fieldCache&stats=true Return info and stats for fieldCache, but nothing else I left the class name alone, but i vote for "SystemInfoRequestHandler" with a default registration of "/admin/info"
        Hide
        Hoss Man added a comment -

        Whoops .. i botched the HTTP Caching prevention in the last version

        Show
        Hoss Man added a comment - Whoops .. i botched the HTTP Caching prevention in the last version
        Hide
        Hoss Man added a comment -

        Committed revision 917812.

        I went ahead and commited the most recent attachment under the name "SystemInfoRequestHandler" with slightly generalized javadocs.

        Leaving the issue open so we make sure to settle the remaining issues before we release...

        • decide if we want to change the name
        • add default registration as part of the AdminRequestHandler (ie: /admin/info ?)
        • add some docs (didn't wnat to make a wiki page until we're certain of hte name)
        • decide if we want to modify the response structure (should all of the top level info be encapsulated in a container?)
        Show
        Hoss Man added a comment - Committed revision 917812. I went ahead and commited the most recent attachment under the name "SystemInfoRequestHandler" with slightly generalized javadocs. Leaving the issue open so we make sure to settle the remaining issues before we release... decide if we want to change the name add default registration as part of the AdminRequestHandler (ie: /admin/info ?) add some docs (didn't wnat to make a wiki page until we're certain of hte name) decide if we want to modify the response structure (should all of the top level info be encapsulated in a container?)
        Hide
        Erik Hatcher added a comment -

        Thanks Hoss for committing!

        naming: I'm fine with how it is, but fine if the name changes too and +1 to adding default registration

        Show
        Erik Hatcher added a comment - Thanks Hoss for committing! naming: I'm fine with how it is, but fine if the name changes too and +1 to adding default registration
        Hide
        Hoss Man added a comment -

        Bulk updating 240 Solr issues to set the Fix Version to "next" per the process outlined in this email...

        http://mail-archives.apache.org/mod_mbox/lucene-dev/201005.mbox/%3Calpine.DEB.1.10.1005251052040.24672@radix.cryptio.net%3E

        Selection criteria was "Unresolved" with a Fix Version of 1.5, 1.6, 3.1, or 4.0. email notifications were suppressed.

        A unique token for finding these 240 issues in the future: hossversioncleanup20100527

        Show
        Hoss Man added a comment - Bulk updating 240 Solr issues to set the Fix Version to "next" per the process outlined in this email... http://mail-archives.apache.org/mod_mbox/lucene-dev/201005.mbox/%3Calpine.DEB.1.10.1005251052040.24672@radix.cryptio.net%3E Selection criteria was "Unresolved" with a Fix Version of 1.5, 1.6, 3.1, or 4.0. email notifications were suppressed. A unique token for finding these 240 issues in the future: hossversioncleanup20100527
        Hide
        Hoss Man added a comment -

        Correcting Fix Version based on CHANGES.txt, see this thread for more details...

        http://mail-archives.apache.org/mod_mbox/lucene-dev/201005.mbox/%3Calpine.DEB.1.10.1005251052040.24672@radix.cryptio.net%3E

        Show
        Hoss Man added a comment - Correcting Fix Version based on CHANGES.txt, see this thread for more details... http://mail-archives.apache.org/mod_mbox/lucene-dev/201005.mbox/%3Calpine.DEB.1.10.1005251052040.24672@radix.cryptio.net%3E
        Hide
        Hoss Man added a comment -

        1) i screwed up, and should have put this in o.a.s.handler.admin instead of o.a.s.handler

        2) Somehow i completely overlooked the fact that there is already a o.a.s.handler.admin.SystemInfoHandler (Erik even mentioned it above) which is registered to the path /admin/system and returns basic info on the current machine, current JVM, the versions of Lucene/Solr, and some basic info about the SolrCore.

        With that in mind i propose we rename this new one to "SolrInfoMBeanHandler" since that's the crux of what it provides (data about all of the SolrInfoMBeans) and have AdminHandler register it with the path /admin/mbeans. We could/should probably also remove some of the code that overlaps between this handler and SystemInfoHandler.

        comments?

        Show
        Hoss Man added a comment - 1) i screwed up, and should have put this in o.a.s.handler.admin instead of o.a.s.handler 2) Somehow i completely overlooked the fact that there is already a o.a.s.handler.admin.SystemInfoHandler (Erik even mentioned it above) which is registered to the path /admin/system and returns basic info on the current machine, current JVM, the versions of Lucene/Solr, and some basic info about the SolrCore. With that in mind i propose we rename this new one to "SolrInfoMBeanHandler" since that's the crux of what it provides (data about all of the SolrInfoMBeans) and have AdminHandler register it with the path /admin/mbeans. We could/should probably also remove some of the code that overlaps between this handler and SystemInfoHandler. comments?
        Hide
        Lance Norskog added a comment -

        Please add an option that just lists the catalog of MBeans.

        Show
        Lance Norskog added a comment - Please add an option that just lists the catalog of MBeans.
        Hide
        Hoss Man added a comment -

        Please add an option that just lists the catalog of MBeans.

        It's already there – if stats=false it just returns the list of SolrInfoMBeans from the registry (like registry.jsp)

        what do you think of the proposed name change & path: SolrInfoMBeanHandler & /admin/mbeans ?

        Show
        Hoss Man added a comment - Please add an option that just lists the catalog of MBeans. It's already there – if stats=false it just returns the list of SolrInfoMBeans from the registry (like registry.jsp) what do you think of the proposed name change & path: SolrInfoMBeanHandler & /admin/mbeans ?
        Hide
        Hoss Man added a comment -

        SOLR-1750 cleanup...

        • rename to o.a.s.handler.admin.SolrInfoMBeanHandler
        • add default registration as part of the AdminRequestHandler /admin/mbeans
        • eliminate duplication of functionality w/SystemInfoHandler
        • "docs" are left in explicit order returned by plugin
        • if "cats" param is used, categories are returned in that order
        Show
        Hoss Man added a comment - SOLR-1750 cleanup... rename to o.a.s.handler.admin.SolrInfoMBeanHandler add default registration as part of the AdminRequestHandler /admin/mbeans eliminate duplication of functionality w/SystemInfoHandler "docs" are left in explicit order returned by plugin if "cats" param is used, categories are returned in that order
        Hide
        Hoss Man added a comment -

        Committed revision 953886. ... trunk
        Committed revision 953887. ... branch 3x

        Show
        Hoss Man added a comment - Committed revision 953886. ... trunk Committed revision 953887. ... branch 3x
        Hide
        Jonathan Rochkind added a comment -

        re: naming. If you're someone like me who is becoming fairly familiar with using solr, but not with the solr code – then "SolrInfoMBeanHandler" or "admin/mbean" doesn't mean anything to me, and is kind of confusing. I want to get info on my indexes and caches-- it would be very non-obvious to me (if i hadn't read this ticket) that "MBean" has anything to do with this, since I don't know what an MBean is – and probably shouldn't have to to use solr through it's APIs.

        So seems to me that a name based on the functions provided (not the underlying internal implementation) is preferable. But i recognize the namespace conflict problems, so much stuff in Solr already (some of it deprecated or soon to be deprecated or removed, some of it not) that it's hard to find a non-conflicting name.

        Even if the underlying class is SolrInfoMBeanHandler, would it be less (or more) confusing for the path to be /admin/info still? That might be less confusing, as someone like me would still see /admin/info in the config and think, aha, that might be what I want. Or the lack of consistency might just be more confusing in the end.

        I don't know what the current SystemInfoHandler does, what's the difference between that and this new one? There might be hints to naming in that. If the new one does everything the old one does, perhaps call it NewSystemInfoHandler, but still register it at /admin/info, with the other one being deprecated? Just brainstorming. Or rename the other one to OldSystemInfoHandler.

        Show
        Jonathan Rochkind added a comment - re: naming. If you're someone like me who is becoming fairly familiar with using solr, but not with the solr code – then "SolrInfoMBeanHandler" or "admin/mbean" doesn't mean anything to me, and is kind of confusing. I want to get info on my indexes and caches-- it would be very non-obvious to me (if i hadn't read this ticket) that "MBean" has anything to do with this, since I don't know what an MBean is – and probably shouldn't have to to use solr through it's APIs. So seems to me that a name based on the functions provided (not the underlying internal implementation) is preferable. But i recognize the namespace conflict problems, so much stuff in Solr already (some of it deprecated or soon to be deprecated or removed, some of it not) that it's hard to find a non-conflicting name. Even if the underlying class is SolrInfoMBeanHandler, would it be less (or more) confusing for the path to be /admin/info still? That might be less confusing, as someone like me would still see /admin/info in the config and think, aha, that might be what I want. Or the lack of consistency might just be more confusing in the end. I don't know what the current SystemInfoHandler does, what's the difference between that and this new one? There might be hints to naming in that. If the new one does everything the old one does, perhaps call it NewSystemInfoHandler, but still register it at /admin/info, with the other one being deprecated? Just brainstorming. Or rename the other one to OldSystemInfoHandler.
        Hide
        Grant Ingersoll added a comment -

        Bulk close for 3.1.0 release

        Show
        Grant Ingersoll added a comment - Bulk close for 3.1.0 release
        Hide
        Jan Høydahl added a comment -

        The /admin/stats handler is not registered by default, nor is it included in example config. I had to add <requestHandler name="/admin/stats" class="org.apache.solr.handler.admin.SolrInfoMBeanHandler" /> to my solrconfig to get it working.

        Show
        Jan Høydahl added a comment - The /admin/stats handler is not registered by default, nor is it included in example config. I had to add <requestHandler name="/admin/stats" class="org.apache.solr.handler.admin.SolrInfoMBeanHandler" /> to my solrconfig to get it working.
        Hide
        Hoss Man added a comment -

        Jan: as stated above the registration i picked was /admin/mbeans - stats is too specific since the component can be used for other purposes then getting stats.

        it's also not a "default" handler – it's registered if you register the AdminHandler

        Jonathan: i overlooked your comment until now. the existing SystemInfoHandler isn't deprecated – it's still very useful and provides information about the entire "system" solr is running in (the jvm, the os, etc...)

        Show
        Hoss Man added a comment - Jan: as stated above the registration i picked was /admin/mbeans - stats is too specific since the component can be used for other purposes then getting stats. it's also not a "default" handler – it's registered if you register the AdminHandler Jonathan: i overlooked your comment until now. the existing SystemInfoHandler isn't deprecated – it's still very useful and provides information about the entire "system" solr is running in (the jvm, the os, etc...)

          People

          • Assignee:
            Erik Hatcher
            Reporter:
            Erik Hatcher
          • Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development