Solr
  1. Solr
  2. SOLR-6779

/browse fails for schemaless example

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 5.0
    • Fix Version/s: 5.0, 6.0
    • Component/s: None
    • Labels:

      Description

      1. bin/solr start -e schemaless
      2. http://localhost:8983/solr/schemaless/browse

      HTTP ERROR 500

      Problem accessing /solr/schemaless/browse. Reason:

      {msg=Unable to find resource 'browse.vm',trace=java.io.IOException: Unable to find resource 'browse.vm'
      at org.apache.solr.response.VelocityResponseWriter.getTemplate(VelocityResponseWriter.java:177)
      at org.apache.solr.response.VelocityResponseWriter.write(VelocityResponseWriter.java:53)
      at org.apache.solr.core.SolrCore$LazyQueryResponseWriterWrapper.write(SolrCore.java:2563)
      at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:53)
      at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:763)
      at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:416)
      at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:202)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
      at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
      at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
      at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
      at org.eclipse.jetty.server.Server.handle(Server.java:368)
      at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
      at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
      at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
      at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
      at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
      at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
      at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
      at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
      at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
      at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
      at java.lang.Thread.run(Thread.java:744)
      ,code=500}

        Activity

        Hide
        ASF subversion and git services added a comment -

        Commit 1641173 from Erik Hatcher in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1641173 ]

        SOLR-6779: Fix /browse for schemaless / data_driven_schema example

        Show
        ASF subversion and git services added a comment - Commit 1641173 from Erik Hatcher in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1641173 ] SOLR-6779 : Fix /browse for schemaless / data_driven_schema example
        Hide
        Erik Hatcher added a comment -

        I've "fixed" this by moving over the original /browse templates and trimming out a ton of stuff. It's a generic document list UI with the same ol' search box. It'll render facet.field's if in the query, but nothing by default (all other faceting/clustering not ported in this initial commit). I'll need help making sure we do the right thing when it comes time to moving this and other such changes to trunk (but I'm not going to Java8 just yet, so trunk is out of reach for me currently).

        The first commit on this was simply to get it working and begin to minimize/genercize the templates. More to do, for sure.

        Show
        Erik Hatcher added a comment - I've "fixed" this by moving over the original /browse templates and trimming out a ton of stuff. It's a generic document list UI with the same ol' search box. It'll render facet.field's if in the query, but nothing by default (all other faceting/clustering not ported in this initial commit). I'll need help making sure we do the right thing when it comes time to moving this and other such changes to trunk (but I'm not going to Java8 just yet, so trunk is out of reach for me currently). The first commit on this was simply to get it working and begin to minimize/genercize the templates. More to do, for sure.
        Hide
        Alexandre Rafalovitch added a comment -

        I was wondering (since this was not a trivial bug, it sounds) whether we still need that handler/velocity at all? It is demonstrating Solr features, true. But, at the same time, it forces us to explain to users that this is not actually a valid production usage. Plus it takes disk space.

        If we need to show-off features, would something Javascript UI based be better instead? Still, "not a production" scenario, but at least no need for another set of libraries.

        Oh, and whatever changes are done to those .vm files, they need to be in all of the examples, right? Because the code is duplicated for each configset.

        Show
        Alexandre Rafalovitch added a comment - I was wondering (since this was not a trivial bug, it sounds) whether we still need that handler/velocity at all? It is demonstrating Solr features, true. But, at the same time, it forces us to explain to users that this is not actually a valid production usage. Plus it takes disk space. If we need to show-off features, would something Javascript UI based be better instead? Still, "not a production" scenario, but at least no need for another set of libraries. Oh, and whatever changes are done to those .vm files, they need to be in all of the examples, right? Because the code is duplicated for each configset.
        Hide
        Uwe Schindler added a comment -

        Hi,
        this was only committed to branch_5x. Is this not intended to be in Solr trunk, too?
        In addition the commit on 5.x caused Jenkins failures because of missing SVN properties for EOL style and Mime-Type.
        Thanks,
        Uwe

        Show
        Uwe Schindler added a comment - Hi, this was only committed to branch_5x. Is this not intended to be in Solr trunk, too? In addition the commit on 5.x caused Jenkins failures because of missing SVN properties for EOL style and Mime-Type. Thanks, Uwe
        Hide
        Jan Høydahl added a comment -

        Alexandre Rafalovitch, yes it would be great to add a pure HTML5 search UI with Solr which could work for all collections automagically, but I feel we are a bit short on UI capable contributors. Have you seen https://github.com/fergiemcdowall/solrstrap ?

        Show
        Jan Høydahl added a comment - Alexandre Rafalovitch , yes it would be great to add a pure HTML5 search UI with Solr which could work for all collections automagically, but I feel we are a bit short on UI capable contributors. Have you seen https://github.com/fergiemcdowall/solrstrap ?
        Hide
        Erik Hatcher added a comment -

        Re: solrstrap, it says 'Designed for "open" solr instances- needs clear access to /select/q=.'

        Show
        Erik Hatcher added a comment - Re: solrstrap, it says 'Designed for "open" solr instances- needs clear access to /select/q=.'
        Hide
        Alexandre Rafalovitch added a comment -

        I don't think Velocity is designed for production environment either, so the use case is probably similar (demo).

        Show
        Alexandre Rafalovitch added a comment - I don't think Velocity is designed for production environment either, so the use case is probably similar (demo).
        Hide
        Erik Hatcher added a comment - - edited

        I don't think Velocity is designed for production environment either, so the use case is probably similar (demo).

        It's really FUD to say that. That's like saying wt=json isn't suited for a production environment. It all really depends on how you use it. I have a first hand account of a huge job search site that uses wt=velocity for generating e-mails; and yeah, that's not generating a direct to users browser view, but it's still "UI" and a quite clever and valid use case.

        No, you should not expose Solr to end users, JSON or XML or any other response format. But that does not mean it's not "for production", again depends on how you define it and use it.

        If one were so inclined, you could front Solr with a proxy/firewall to prevent updates or other malicious or unintended uses and have the UI come from wt=velocity even. Totally production quality in that case.

        Show
        Erik Hatcher added a comment - - edited I don't think Velocity is designed for production environment either, so the use case is probably similar (demo). It's really FUD to say that. That's like saying wt=json isn't suited for a production environment. It all really depends on how you use it. I have a first hand account of a huge job search site that uses wt=velocity for generating e-mails; and yeah, that's not generating a direct to users browser view, but it's still "UI" and a quite clever and valid use case. No, you should not expose Solr to end users, JSON or XML or any other response format. But that does not mean it's not "for production", again depends on how you define it and use it. If one were so inclined, you could front Solr with a proxy/firewall to prevent updates or other malicious or unintended uses and have the UI come from wt=velocity even. Totally production quality in that case.
        Hide
        Alexandre Rafalovitch added a comment -

        No FUD intended.

        However, we have users putting Solr facing the web directly and we have to slap their hands all the time. So, the easy message is No SOLR facing the web. The nuanced message is , unless you know what you are doing and have smart proxy setup. But there is no easy official example of that smart proxy/firewall rules (some random GitHub configs do exists), so it is not fair to give a nuanced answer without holding the hand on making it happen.

        Show
        Alexandre Rafalovitch added a comment - No FUD intended. However, we have users putting Solr facing the web directly and we have to slap their hands all the time. So, the easy message is No SOLR facing the web . The nuanced message is , unless you know what you are doing and have smart proxy setup . But there is no easy official example of that smart proxy/firewall rules (some random GitHub configs do exists), so it is not fair to give a nuanced answer without holding the hand on making it happen.
        Hide
        Erik Hatcher added a comment -

        However, we have users putting Solr facing the web directly...

        and putting HTML/JavaScript/Ajax UI's on top of Solr without security too. Again, wt=velocity is no different than wt=json.

        Point taken, of course, just wanted to, again, attempt to dispel what I can about /browse. Certainly it was intended as a prototype / playground for the techproducts data example, but it's more than just that. One of the things I'm working on and alluded to in e-mail recently was to pull the templates out of conf/velocity and get basic ones baked into either the solr-velocity JAR or into a separate JAR. It's probably little known that wt=velocity can serve templates from the SolrResourceLoader (i.e. the plugins). One of the goals has always been to have plugins be able to have their own UI bits, and wt=velocity has supported that since day one. I'll do more to showcase this in 5.x (hopefully in time for 5.0 even), with template overridability (already supported) too. More on this in a few weeks.

        Show
        Erik Hatcher added a comment - However, we have users putting Solr facing the web directly... and putting HTML/JavaScript/Ajax UI's on top of Solr without security too. Again, wt=velocity is no different than wt=json. Point taken, of course, just wanted to, again, attempt to dispel what I can about /browse. Certainly it was intended as a prototype / playground for the techproducts data example, but it's more than just that. One of the things I'm working on and alluded to in e-mail recently was to pull the templates out of conf/velocity and get basic ones baked into either the solr-velocity JAR or into a separate JAR. It's probably little known that wt=velocity can serve templates from the SolrResourceLoader (i.e. the plugins). One of the goals has always been to have plugins be able to have their own UI bits, and wt=velocity has supported that since day one. I'll do more to showcase this in 5.x (hopefully in time for 5.0 even), with template overridability (already supported) too. More on this in a few weeks.
        Hide
        Alexandre Rafalovitch added a comment -

        It's probably little known that wt=velocity can serve templates from the SolrResourceLoader (i.e. the plugins).

        Well, it's news to me. An exciting news for sure. Is there a blog post or documentation with an example of how to do that? Even a basic one showing the directory layout or whatever.

        Show
        Alexandre Rafalovitch added a comment - It's probably little known that wt=velocity can serve templates from the SolrResourceLoader (i.e. the plugins). Well, it's news to me. An exciting news for sure. Is there a blog post or documentation with an example of how to do that? Even a basic one showing the directory layout or whatever.
        Hide
        Erik Hatcher added a comment -

        Is there a blog post or documentation with an example of how to do that?

        Not really, but locally on my 5x branch I simply moved the templates from server/solr/configsets/data_driven_schema_configs/conf/velocity to contrib/velocity/src/resources/velocity/ (a new directory), which gets the .vm files into the solr-velocity JAR file (under a velocity/ sub-tree in the JAR structure) and rebuilt Solr and /browse still works fine.

        Actually, the SolrResourceLoader picks up anything in conf/ or within JAR files automatically, so it's really nothing special that the VelocityResponseWriter is doing to support this.

        For the history, this is when it was added: https://issues.apache.org/jira/browse/SOLR-620?focusedCommentId=12644491&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12644491

        Show
        Erik Hatcher added a comment - Is there a blog post or documentation with an example of how to do that? Not really, but locally on my 5x branch I simply moved the templates from server/solr/configsets/data_driven_schema_configs/conf/velocity to contrib/velocity/src/resources/velocity/ (a new directory), which gets the .vm files into the solr-velocity JAR file (under a velocity/ sub-tree in the JAR structure) and rebuilt Solr and /browse still works fine. Actually, the SolrResourceLoader picks up anything in conf/ or within JAR files automatically, so it's really nothing special that the VelocityResponseWriter is doing to support this. For the history, this is when it was added: https://issues.apache.org/jira/browse/SOLR-620?focusedCommentId=12644491&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12644491
        Hide
        ASF subversion and git services added a comment -

        Commit 1648028 from Erik Hatcher in branch 'dev/trunk'
        [ https://svn.apache.org/r1648028 ]

        SOLR-6779: remove techproduct-specific settings from generic /browse

        Show
        ASF subversion and git services added a comment - Commit 1648028 from Erik Hatcher in branch 'dev/trunk' [ https://svn.apache.org/r1648028 ] SOLR-6779 : remove techproduct-specific settings from generic /browse
        Hide
        ASF subversion and git services added a comment -

        Commit 1648030 from Erik Hatcher in branch 'dev/trunk'
        [ https://svn.apache.org/r1648030 ]

        SOLR-6779: add trimmed /browse templates to Velocity contrib JAR, making them available to any VrW using system

        Show
        ASF subversion and git services added a comment - Commit 1648030 from Erik Hatcher in branch 'dev/trunk' [ https://svn.apache.org/r1648030 ] SOLR-6779 : add trimmed /browse templates to Velocity contrib JAR, making them available to any VrW using system
        Hide
        ASF subversion and git services added a comment -

        Commit 1648031 from Erik Hatcher in branch 'dev/trunk'
        [ https://svn.apache.org/r1648031 ]

        SOLR-6779: fix svn:eol-style

        Show
        ASF subversion and git services added a comment - Commit 1648031 from Erik Hatcher in branch 'dev/trunk' [ https://svn.apache.org/r1648031 ] SOLR-6779 : fix svn:eol-style
        Hide
        Erik Hatcher added a comment -

        This has been corrected on trunk. I'm going to do a number of more changes in this area and will merge it all to 5x when done on trunk.

        With this change, the default templates are housed in the solr-velocity JAR (under velocity/ path) and are available to any system with VelocityResponseWriter registered. These templates can be used without registering a new /browse-like request handler, for example: /select?wt=velocity&v.template=browse&v.layout=layout.

        [I also snuck an _default.vm template in to my last commit that currently isn't referenced directly anywhere, with a later VrW refactoring to make that the default if v.template isn't specified]

        Show
        Erik Hatcher added a comment - This has been corrected on trunk. I'm going to do a number of more changes in this area and will merge it all to 5x when done on trunk. With this change, the default templates are housed in the solr-velocity JAR (under velocity/ path) and are available to any system with VelocityResponseWriter registered. These templates can be used without registering a new /browse-like request handler, for example: /select?wt=velocity&v.template=browse&v.layout=layout. [I also snuck an _default.vm template in to my last commit that currently isn't referenced directly anywhere, with a later VrW refactoring to make that the default if v.template isn't specified]
        Hide
        ASF subversion and git services added a comment -

        Commit 1648456 from Erik Hatcher in branch 'dev/trunk'
        [ https://svn.apache.org/r1648456 ]

        SOLR-6779 (fix /browse on schemaless example) and several other related fixes and cleanups including remove /browse from test configs where it isn't needed, adjusting /browse definition to be valid and not an inappropriate copy of techproducts, fixed /update/extract indexing with data_driven_schema_configs configset, and fixed indexing of solr(-to-solr) example DIH indexing

        Show
        ASF subversion and git services added a comment - Commit 1648456 from Erik Hatcher in branch 'dev/trunk' [ https://svn.apache.org/r1648456 ] SOLR-6779 (fix /browse on schemaless example) and several other related fixes and cleanups including remove /browse from test configs where it isn't needed, adjusting /browse definition to be valid and not an inappropriate copy of techproducts, fixed /update/extract indexing with data_driven_schema_configs configset, and fixed indexing of solr(-to-solr) example DIH indexing
        Hide
        ASF subversion and git services added a comment -

        Commit 1648459 from Erik Hatcher in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1648459 ]

        SOLR-6779: much /browse cleanup, merged from trunk r1648028, r1648030, and r1648031

        Show
        ASF subversion and git services added a comment - Commit 1648459 from Erik Hatcher in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1648459 ] SOLR-6779 : much /browse cleanup, merged from trunk r1648028, r1648030, and r1648031
        Hide
        ASF subversion and git services added a comment -

        Commit 1648463 from Erik Hatcher in branch 'dev/trunk'
        [ https://svn.apache.org/r1648463 ]

        SOLR-6779: adjust trunk CHANGES to reflect the merge to 5x

        Show
        ASF subversion and git services added a comment - Commit 1648463 from Erik Hatcher in branch 'dev/trunk' [ https://svn.apache.org/r1648463 ] SOLR-6779 : adjust trunk CHANGES to reflect the merge to 5x
        Hide
        ASF subversion and git services added a comment -

        Commit 1648464 from Erik Hatcher in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1648464 ]

        SOLR-6779: remove directory that should have been removed with previous merge

        Show
        ASF subversion and git services added a comment - Commit 1648464 from Erik Hatcher in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1648464 ] SOLR-6779 : remove directory that should have been removed with previous merge
        Hide
        Erik Hatcher added a comment -

        Fixed, and then some.

        Show
        Erik Hatcher added a comment - Fixed, and then some.
        Hide
        Anshum Gupta added a comment -

        Bulk close after 5.0 release.

        Show
        Anshum Gupta added a comment - Bulk close after 5.0 release.

          People

          • Assignee:
            Erik Hatcher
            Reporter:
            Alexandre Rafalovitch
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development