Solr
  1. Solr
  2. SOLR-1506

Search multiple cores using MultiReader

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Trivial Trivial
    • Resolution: Unresolved
    • Affects Version/s: 1.4
    • Fix Version/s: 4.9, Trunk
    • Component/s: search
    • Labels:
      None

      Description

      I need to search over multiple cores, and SOLR-1477 is more
      complicated than expected, so here we'll create a MultiReader
      over the cores to allow searching on them.

      Maybe in the future we can add parallel searching however
      SOLR-1477, if it gets completed, provides that out of the box.

      1. SOLR-1506.patch
        10 kB
        Jason Rutherglen
      2. SOLR-1506.patch
        12 kB
        Jason Rutherglen
      3. SOLR-1506.patch
        13 kB
        Jason Rutherglen
      4. TIKA.txt
        0.5 kB
        anil Sharma

        Activity

        Hide
        Jason Rutherglen added a comment -

        Well, it seems to work, though I had to comment out the reader.directory() call in SolrCore. I'm not sure what to do there yet, but this is good enough for now.

        Show
        Jason Rutherglen added a comment - Well, it seems to work, though I had to comment out the reader.directory() call in SolrCore. I'm not sure what to do there yet, but this is good enough for now.
        Hide
        Jason Rutherglen added a comment -

        Fixes a bug, added Apache headers

        Show
        Jason Rutherglen added a comment - Fixes a bug, added Apache headers
        Hide
        Jason Rutherglen added a comment -

        There's a bug here with getting the status of multiple cores:

        SEVERE: org.apache.solr.common.SolrException: Error handling 'status' action
        at org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:362)
        at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:131)
        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
        at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:298)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:174)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
        at java.lang.Thread.run(Thread.java:619)
        Caused by: java.lang.UnsupportedOperationException: This reader does not support this method.
        at org.apache.lucene.index.IndexReader.directory(IndexReader.java:592)
        at org.apache.solr.search.SolrIndexReader.directory(SolrIndexReader.java:222)
        at org.apache.solr.handler.admin.LukeRequestHandler.getIndexInfo(LukeRequestHandler.java:442)
        at org.apache.solr.handler.admin.CoreAdminHandler.getCoreStatus(CoreAdminHandler.java:449)
        at org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:353

        Show
        Jason Rutherglen added a comment - There's a bug here with getting the status of multiple cores: SEVERE: org.apache.solr.common.SolrException: Error handling 'status' action at org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:362) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:131) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:298) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:174) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454) at java.lang.Thread.run(Thread.java:619) Caused by: java.lang.UnsupportedOperationException: This reader does not support this method. at org.apache.lucene.index.IndexReader.directory(IndexReader.java:592) at org.apache.solr.search.SolrIndexReader.directory(SolrIndexReader.java:222) at org.apache.solr.handler.admin.LukeRequestHandler.getIndexInfo(LukeRequestHandler.java:442) at org.apache.solr.handler.admin.CoreAdminHandler.getCoreStatus(CoreAdminHandler.java:449) at org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:353
        Hide
        Jason Rutherglen added a comment -

        Commit doesn't work because reopen isn't supported by MultiReader.

        Show
        Jason Rutherglen added a comment - Commit doesn't work because reopen isn't supported by MultiReader.
        Hide
        Koji Sekiguchi added a comment -

        Commit doesn't work because reopen isn't supported by MultiReader.

        Regarding MultiReader and reopen, I've set reopenReaders to false:

        solrconfig.xml
        <reopenReaders>false</reopenReaders>
          :
        <indexReaderFactory name="IndexReaderFactory" class="mypackage.MultiReaderFactory"/>
        
        Show
        Koji Sekiguchi added a comment - Commit doesn't work because reopen isn't supported by MultiReader. Regarding MultiReader and reopen, I've set reopenReaders to false: solrconfig.xml <reopenReaders> false </reopenReaders> : <indexReaderFactory name= "IndexReaderFactory" class= "mypackage.MultiReaderFactory" />
        Hide
        Jason Rutherglen added a comment -

        MultiReader doesn't support reopen with the readOnly parameter. This patch adds a test case for commit on the proxy, and a workaround (if unsupported is caught, then regular reopen is called).

        Show
        Jason Rutherglen added a comment - MultiReader doesn't support reopen with the readOnly parameter. This patch adds a test case for commit on the proxy, and a workaround (if unsupported is caught, then regular reopen is called).
        Hide
        Jason Rutherglen added a comment -

        There's a different bug here, where because CoreContainer loads
        the cores sequentially, and MultiCoreReaderFactory looks for all
        the cores, when the proxy core isn't last, not all the cores are
        searchable, if the proxy is first, an exception is thrown.

        The workaround is to place the proxy core last, however that's
        not possible when using the core admin HTTP API. Hmm... Not sure
        what the best workaround is.

        Show
        Jason Rutherglen added a comment - There's a different bug here, where because CoreContainer loads the cores sequentially, and MultiCoreReaderFactory looks for all the cores, when the proxy core isn't last, not all the cores are searchable, if the proxy is first, an exception is thrown. The workaround is to place the proxy core last, however that's not possible when using the core admin HTTP API. Hmm... Not sure what the best workaround is.
        Hide
        Hoss Man added a comment -

        Bulk updating 240 Solr issues to set the Fix Version to "next" per the process outlined in this email...

        http://mail-archives.apache.org/mod_mbox/lucene-dev/201005.mbox/%3Calpine.DEB.1.10.1005251052040.24672@radix.cryptio.net%3E

        Selection criteria was "Unresolved" with a Fix Version of 1.5, 1.6, 3.1, or 4.0. email notifications were suppressed.

        A unique token for finding these 240 issues in the future: hossversioncleanup20100527

        Show
        Hoss Man added a comment - Bulk updating 240 Solr issues to set the Fix Version to "next" per the process outlined in this email... http://mail-archives.apache.org/mod_mbox/lucene-dev/201005.mbox/%3Calpine.DEB.1.10.1005251052040.24672@radix.cryptio.net%3E Selection criteria was "Unresolved" with a Fix Version of 1.5, 1.6, 3.1, or 4.0. email notifications were suppressed. A unique token for finding these 240 issues in the future: hossversioncleanup20100527
        Hide
        anil Sharma added a comment -

        stwte

        Show
        anil Sharma added a comment - stwte
        Hide
        Robert Muir added a comment -

        Bulk move 3.2 -> 3.3

        Show
        Robert Muir added a comment - Bulk move 3.2 -> 3.3
        Hide
        Robert Muir added a comment -

        3.4 -> 3.5

        Show
        Robert Muir added a comment - 3.4 -> 3.5
        Hide
        Hoss Man added a comment -

        Bulk of fixVersion=3.6 -> fixVersion=4.0 for issues that have no assignee and have not been updated recently.

        email notification suppressed to prevent mass-spam
        psuedo-unique token identifying these issues: hoss20120321nofix36

        Show
        Hoss Man added a comment - Bulk of fixVersion=3.6 -> fixVersion=4.0 for issues that have no assignee and have not been updated recently. email notification suppressed to prevent mass-spam psuedo-unique token identifying these issues: hoss20120321nofix36
        Hide
        Steve Rowe added a comment -

        Bulk move 4.4 issues to 4.5 and 5.0

        Show
        Steve Rowe added a comment - Bulk move 4.4 issues to 4.5 and 5.0
        Hide
        Uwe Schindler added a comment -

        Move issue to Solr 4.9.

        Show
        Uwe Schindler added a comment - Move issue to Solr 4.9.

          People

          • Assignee:
            Unassigned
            Reporter:
            Jason Rutherglen
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:

              Development