Solr
  1. Solr
  2. SOLR-2824

Cross-Core Join doesn't parse fields against joining schema

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 4.0-ALPHA
    • Fix Version/s: 4.0-ALPHA
    • Component/s: multicore, search
    • Labels:
      None

      Description

      I have two cores with 2 different schema's
      now I want to join between the 2 cores. where I filter on a field from one core that doesn't exist in the other core.
      core1:

      {childIds, name, id}

      , core2:

      {id, type, specials}

      I have the following query
      /core1/select?q=:&fq=

      {!join from=id to=childIds fromIndex=core2}

      specials:1&fl=id,name
      I get this exception [1]

      Looking at the debugger I see that
      at org.apache.solr.search.JoinQParserPlugin$1.parse(JoinQParserPlugin.java:60)
      the parse is called for the filterquery on the main core (core1). Not the core of the 'fromIndex' (core2)

      [1]
      SEVERE: org.apache.solr.common.SolrException: undefined field specials
      at org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1028)
      at org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getWrappedAnalyzer(IndexSchema.java:335)
      at org.apache.lucene.analysis.AnalyzerWrapper.createComponents(AnalyzerWrapper.java:71)
      at org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:83)
      at org.apache.lucene.queryparser.classic.QueryParserBase.newFieldQuery(QueryParserBase.java:476)
      at org.apache.lucene.queryparser.classic.QueryParserBase.getFieldQuery(QueryParserBase.java:464)
      at org.apache.solr.search.SolrQueryParser.getFieldQuery(SolrQueryParser.java:134)
      at org.apache.lucene.queryparser.classic.QueryParserBase.handleBareTokenQuery(QueryParserBase.java:1052)
      at org.apache.lucene.queryparser.classic.QueryParser.Term(QueryParser.java:358)
      at org.apache.lucene.queryparser.classic.QueryParser.Clause(QueryParser.java:257)
      at org.apache.lucene.queryparser.classic.QueryParser.Query(QueryParser.java:181)
      at org.apache.lucene.queryparser.classic.QueryParser.TopLevelQuery(QueryParser.java:170)
      at org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:118)
      at org.apache.solr.search.LuceneQParser.parse(LuceneQParserPlugin.java:74)
      at org.apache.solr.search.QParser.getQuery(QParser.java:143)
      at org.apache.solr.search.JoinQParserPlugin$1.parse(JoinQParserPlugin.java:60)
      at org.apache.solr.search.QParser.getQuery(QParser.java:143)
      at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:138)
      at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:180)
      at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
      at org.apache.solr.core.SolrCore.execute(SolrCore.java:1452)
      at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:353)
      at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
      at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
      at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
      at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
      at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
      at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
      at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
      at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
      at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
      at java.lang.Thread.run(Thread.java:662)

      1. SOLR-2824.patch
        5 kB
        Yonik Seeley
      2. SOLR-2824.patch
        5 kB
        Hoss Man

        Activity

        Hide
        Yonik Seeley added a comment -

        Here's a patch for the last big issue with cross-join: caching of the join when the fromCore searcher has changed.

        Show
        Yonik Seeley added a comment - Here's a patch for the last big issue with cross-join: caching of the join when the fromCore searcher has changed.
        Hide
        Yonik Seeley added a comment -

        I just checked in a patch to parse the query in the correct core.

        Show
        Yonik Seeley added a comment - I just checked in a patch to parse the query in the correct core.
        Hide
        Yonik Seeley added a comment -

        I just checked in a patch for the rewrite issue. Still need a patch to parse the query in the correct core though.

        Show
        Yonik Seeley added a comment - I just checked in a patch for the rewrite issue. Still need a patch to parse the query in the correct core though.
        Hide
        Thijs Vonk added a comment -

        I hope you don't remove it.
        I'm currently faking it by having the 'joined' columns listed in both schema-files and that works.

        Show
        Thijs Vonk added a comment - I hope you don't remove it. I'm currently faking it by having the 'joined' columns listed in both schema-files and that works.
        Hide
        Hoss Man added a comment -

        marking blocker for 4.0 ... if no one comes up with a fix for this problem in time for 4.0, then we should remove the cross-core join code (it's never been documented, and it's never really worked properly except in extremely trivial situations)

        Show
        Hoss Man added a comment - marking blocker for 4.0 ... if no one comes up with a fix for this problem in time for 4.0, then we should remove the cross-core join code (it's never been documented, and it's never really worked properly except in extremely trivial situations)
        Hide
        Yonik Seeley added a comment -

        1) should local params from the '{!join...}' expression (besides from,to, and fromIndex) be passed to the QParser fetched from the fromIndex ?

        defType is the only local param that would have an impact on the foreign qparser.
        But... all the request parameters should be available to the foreign qparser.

        b1. 2) Are there other bugs lurking in JoinQuery itself that need better cross-index testing? In particular the JoinQuery.rewrite method smells fishy.

        Yep!

        Show
        Yonik Seeley added a comment - 1) should local params from the '{!join...}' expression (besides from,to, and fromIndex) be passed to the QParser fetched from the fromIndex ? defType is the only local param that would have an impact on the foreign qparser. But... all the request parameters should be available to the foreign qparser. b1. 2) Are there other bugs lurking in JoinQuery itself that need better cross-index testing? In particular the JoinQuery.rewrite method smells fishy. Yep!
        Hide
        Hoss Man added a comment -

        I hacked up a test case demonstrating the problem, but ran out of time before i could try fixing it. (At the moment the test just demonstrates the problem of parsing a cross index join, it doesn't actually try to utilize it - our Test framework does't make it easy to index/query multiple cores in a single test)

        At first glance, it seems fairly straight forward to fix the JoinQParser to get the "fromIndex" SolrCore via the CoreContainer and then ask it for a parser to parse the nested query, but a few things concern me:

        1) should local params from the '

        {!join...}

        ' expression (besides from,to, and fromIndex) be passed to the QParser fetched from the fromIndex ?

        2) Are there other bugs lurking in JoinQuery itself that need better cross-index testing? In particular the JoinQuery.rewrite method smells fishy.

        Show
        Hoss Man added a comment - I hacked up a test case demonstrating the problem, but ran out of time before i could try fixing it. (At the moment the test just demonstrates the problem of parsing a cross index join, it doesn't actually try to utilize it - our Test framework does't make it easy to index/query multiple cores in a single test) At first glance, it seems fairly straight forward to fix the JoinQParser to get the "fromIndex" SolrCore via the CoreContainer and then ask it for a parser to parse the nested query, but a few things concern me: 1) should local params from the ' {!join...} ' expression (besides from,to, and fromIndex) be passed to the QParser fetched from the fromIndex ? 2) Are there other bugs lurking in JoinQuery itself that need better cross-index testing? In particular the JoinQuery.rewrite method smells fishy.
        Hide
        Yonik Seeley added a comment -

        You are correct... this is a bug because we should parse the nested query in the other core.

        Show
        Yonik Seeley added a comment - You are correct... this is a bug because we should parse the nested query in the other core.

          People

          • Assignee:
            Unassigned
            Reporter:
            Thijs Vonk
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development