Solr
  1. Solr
  2. SOLR-6234

Scoring modes for query time join

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 5.3
    • Fix Version/s: 5.3
    • Component/s: query parsers
    • Labels:

      Description

      it adds ability to call Lucene's JoinUtil by specifying local param, ie {!join score=...}.... It supports:

      • score=none|avg|max|total local param (passed as ScoreMode to JoinUtil)
      • supports b=100 param to pass Query.setBoost() postponed till SOLR-7814.
      • multiVals=true|false is introduced YAGNI, let me know otherwise.
      • there is a test coverage for cross core join case.
      • so far it joins string and multivalue string fields (Sorted, SortedSet, Binary), but not Numerics DVs. follow-up LUCENE-5868
      1. otherHandler.patch
        3 kB
        Ryan Josal
      2. SOLR-6234.patch
        58 kB
        Mikhail Khludnev
      3. SOLR-6234.patch
        57 kB
        Mikhail Khludnev
      4. SOLR-6234.patch
        57 kB
        Mikhail Khludnev
      5. SOLR-6234.patch
        53 kB
        Mikhail Khludnev
      6. SOLR-6234.patch
        53 kB
        Timothy Potter
      7. SOLR-6234.patch
        52 kB
        Mikhail Khludnev
      8. SOLR-6234.patch
        46 kB
        Mikhail Khludnev
      9. SOLR-6234-javadocsfix.patch
        3 kB
        Uwe Schindler
      10. SOLR-6234-trunk-CHANGES-fix.patch
        0.9 kB
        Mikhail Khludnev

        Issue Links

          Activity

          Hide
          Jack Lo added a comment -

          It would be better to integrate this into the main JoinQParser where no scoring would just be the old and default behavior. And since you are using Lucene JoinUtils, we can get rid of the old code in JoinQParser as well.
          I also have a patch that partially make join work in a distributed environment. It's in SOLR-4905, but that's reusing code in JoinQParser and doesn't work if fromIndex has more than 1 shard. However, by fixing LUCENE-3759 and combining your patch, we can make distributed join work as long as all shards reside locally.

          Show
          Jack Lo added a comment - It would be better to integrate this into the main JoinQParser where no scoring would just be the old and default behavior. And since you are using Lucene JoinUtils, we can get rid of the old code in JoinQParser as well. I also have a patch that partially make join work in a distributed environment. It's in SOLR-4905 , but that's reusing code in JoinQParser and doesn't work if fromIndex has more than 1 shard. However, by fixing LUCENE-3759 and combining your patch, we can make distributed join work as long as all shards reside locally.
          Hide
          Mikhail Khludnev added a comment -

          Jack Lo
          I still think we need to add this QParse for Solr users, rather than decommission the current Solr join. I agree its' code is not easy to read, but I suppose it performs better in certain cases, and/or consume fewer memory than straightforward JoinUtil.

          re LUCENE-3759 : I don't believe that true distributed join performs well for practical usage (here I agree with Yonik's comment at SOLR-4905). As far as I understand, what you've done in SOLR-4905 allows to do collocated join even with multiple shards ie. if we place from and to side documents at the same shard, it's able to join them. I think it's what we need.

          Show
          Mikhail Khludnev added a comment - Jack Lo I still think we need to add this QParse for Solr users, rather than decommission the current Solr join . I agree its' code is not easy to read, but I suppose it performs better in certain cases, and/or consume fewer memory than straightforward JoinUtil. re LUCENE-3759 : I don't believe that true distributed join performs well for practical usage (here I agree with Yonik's comment at SOLR-4905 ). As far as I understand, what you've done in SOLR-4905 allows to do collocated join even with multiple shards ie. if we place from and to side documents at the same shard, it's able to join them. I think it's what we need.
          Hide
          Mikhail Khludnev added a comment -

          Martijn van Groningen Robert Muir let me gently remind you about this patch.
          As a side note, this ticket allows to reveal LUCENE-5653, wouldn't you mind to provide scorejoin for Solr users!

          Show
          Mikhail Khludnev added a comment - Martijn van Groningen Robert Muir let me gently remind you about this patch. As a side note, this ticket allows to reveal LUCENE-5653 , wouldn't you mind to provide scorejoin for Solr users!
          Hide
          Mikhail Khludnev added a comment -

          formatted the patch. Still there is no anyone who'd like to have Lucene join with scoring available in Solr.

          Show
          Mikhail Khludnev added a comment - formatted the patch. Still there is no anyone who'd like to have Lucene join with scoring available in Solr.
          Hide
          Mikhail Khludnev added a comment -

          damn, the last patch doesn't use multiVals as a component at equals/hashcode. It's not a big deal, because the param itself seems pretty useless. Anyway, I can fixed it in a few mins, if you want to apply the patch.

          Show
          Mikhail Khludnev added a comment - damn, the last patch doesn't use multiVals as a component at equals/hashcode. It's not a big deal, because the param itself seems pretty useless. Anyway, I can fixed it in a few mins, if you want to apply the patch.
          Hide
          Mikhail Khludnev added a comment -

          Use account "steve_rowe" instead do you mean this one?

          Show
          Mikhail Khludnev added a comment - Use account "steve_rowe" instead do you mean this one?
          Hide
          Mikhail Khludnev added a comment -

          Erik Hatcher this one is great for 4.10.3

          Show
          Mikhail Khludnev added a comment - Erik Hatcher this one is great for 4.10.3
          Hide
          Parnit added a comment -

          Hi Mikhail,
          I emailed a few mailing lists and you replied to help solve my scenario.
          I am slightly confused by how this feature would help also would you be able to provide sample usage for my scenario?
          Here it is in case you had forgotten.

          I have my mainIndex which looks something like this.
          id description name
          1 "description1 " "name1"
          2 "description2 " "name2"
          I also have a subIndex which looks something like this
          id metric
          1 4
          2 5
          What I am trying to do is join the two index's on the id column and have my results sorted based on a column from the subIndex with a query like the following.

          testServer/solr/MainIndex/select?defType=edismax&q=*&fq=

          {!join from=id to=id fromIndex=subIndex}

          id:*&sort=metric desc
          desired result
          2 "description2 " "name2"
          1 "description1 " "name1"
          I'm aware that you lose all information from the subIndex the moment the parser sees "&". What are my options should I want to join two indexes and sort on a column not present in the main index?

          If you or someone else could show me some sample usage I would greatly appreciate it.

          Thanks

          Show
          Parnit added a comment - Hi Mikhail, I emailed a few mailing lists and you replied to help solve my scenario. I am slightly confused by how this feature would help also would you be able to provide sample usage for my scenario? Here it is in case you had forgotten. I have my mainIndex which looks something like this. id description name 1 "description1 " "name1" 2 "description2 " "name2" I also have a subIndex which looks something like this id metric 1 4 2 5 What I am trying to do is join the two index's on the id column and have my results sorted based on a column from the subIndex with a query like the following. testServer/solr/MainIndex/select?defType=edismax&q=*&fq= {!join from=id to=id fromIndex=subIndex} id:*&sort=metric desc desired result 2 "description2 " "name2" 1 "description1 " "name1" I'm aware that you lose all information from the subIndex the moment the parser sees "&". What are my options should I want to join two indexes and sort on a column not present in the main index? If you or someone else could show me some sample usage I would greatly appreciate it. Thanks
          Hide
          Mikhail Khludnev added a comment -

          lose all information from the subIndex the moment the parser sees "&".

          that's what I don't understand.

          nevertheless it should work by supplying field function query as a "to" side query

          testServer/solr/MainIndex/select?q=

          {!scorejoin from=id to=id fromIndex=subIndex}

          metric

          Show
          Mikhail Khludnev added a comment - lose all information from the subIndex the moment the parser sees "&". that's what I don't understand. nevertheless it should work by supplying field function query as a "to" side query testServer/solr/MainIndex/select?q= {!scorejoin from=id to=id fromIndex=subIndex} metric
          Hide
          Parnit added a comment -

          thanks for the example, what I meant by that was when you supply the "&" you go back to querying the main index and no longer have access to fields from the subindex.
          Also I'm assuming a negative query will give desc or asc order so I would be able to support both orders?

          Show
          Parnit added a comment - thanks for the example, what I meant by that was when you supply the "&" you go back to querying the main index and no longer have access to fields from the subindex. Also I'm assuming a negative query will give desc or asc order so I would be able to support both orders?
          Hide
          Mikhail Khludnev added a comment -

          I'm sure &sort=score asc&.. will reverse results

          Show
          Mikhail Khludnev added a comment - I'm sure &sort=score asc&.. will reverse results
          Hide
          Parnit added a comment -

          I have been getting some strange behaviour using scorejoin
          testServer/solr/MainIndex/select?q=

          {!squarejoin from=id to=id fromIndex=subIndex}

          metric:*
          returns the following error
          undefined field metric

          If I try the same with Solr join I am able to access column metric just fine.
          testServer/solr/MainIndex/select?q=

          {!join from=id to=id fromIndex=subIndex}

          metric:*
          If I try using scorejoin with a column that exists in the mainIndex it seems to be returning results.
          Is this a bug?

          Show
          Parnit added a comment - I have been getting some strange behaviour using scorejoin testServer/solr/MainIndex/select?q= {!squarejoin from=id to=id fromIndex=subIndex} metric:* returns the following error undefined field metric If I try the same with Solr join I am able to access column metric just fine. testServer/solr/MainIndex/select?q= {!join from=id to=id fromIndex=subIndex} metric:* If I try using scorejoin with a column that exists in the mainIndex it seems to be returning results. Is this a bug?
          Hide
          Mikhail Khludnev added a comment -

          Is this a bug.

          yes it is. the parsing of fromQuery is incorrect in case of cross core join.
          Shame on me. The only excuse is the absence of two cores tests in overall test codebase.

          Committers, is there an example of bi-core test?

          Show
          Mikhail Khludnev added a comment - Is this a bug. yes it is. the parsing of fromQuery is incorrect in case of cross core join. Shame on me. The only excuse is the absence of two cores tests in overall test codebase. Committers, is there an example of bi-core test?
          Hide
          Mikhail Khludnev added a comment -

          Parnit I have a workaround. The problem occurs only during query parsing. I suppose you can just define optional 'metric' field in the MainIndex that allows to parse fromQuery, and come further.
          Pls post exception if it doesn't help you.

          Show
          Mikhail Khludnev added a comment - Parnit I have a workaround. The problem occurs only during query parsing. I suppose you can just define optional 'metric' field in the MainIndex that allows to parse fromQuery, and come further. Pls post exception if it doesn't help you.
          Hide
          Parnit added a comment -

          Hi Mikhail, I tried what you are suggesting although it no longer throws an exception I believe it is not working as intended.
          Given the hypothetical situation where I add a new metric to subIndex lets say metric2
          so something like this

          id metric1 metric2
          1 4 7
          2 5 2

          testServer/solr/MC_10001_CatalogEntry_en_US/select?q=

          {!scorejoin from=id to=id fromIndex=subIndex}metric1:*
          returns
          doc1,
          doc2
          from the main index
          That leads me to believe the scoreJoin is returning results in ascending order.

          now the following
          testServer/solr/MC_10001_CatalogEntry_en_US/select?q={!scorejoin from=id to=id fromIndex=subIndex}

          metric2:*
          also returns
          doc1,
          doc2
          from the main index
          the metric appears to have no impact on the ordering of the results returned.
          Lastly ideal situation is not having to specify metric in both mainIndex and subIndex as optional as "metric" columns are dynamic columns in subIndex, it would be extremely inconvenient to add it to the mainIndex as well each time a new column is added.

          Thanks for the continued help.

          Show
          Parnit added a comment - Hi Mikhail, I tried what you are suggesting although it no longer throws an exception I believe it is not working as intended. Given the hypothetical situation where I add a new metric to subIndex lets say metric2 so something like this id metric1 metric2 1 4 7 2 5 2 testServer/solr/MC_10001_CatalogEntry_en_US/select?q= {!scorejoin from=id to=id fromIndex=subIndex}metric1:* returns doc1, doc2 from the main index That leads me to believe the scoreJoin is returning results in ascending order. now the following testServer/solr/MC_10001_CatalogEntry_en_US/select?q={!scorejoin from=id to=id fromIndex=subIndex} metric2:* also returns doc1, doc2 from the main index the metric appears to have no impact on the ordering of the results returned. Lastly ideal situation is not having to specify metric in both mainIndex and subIndex as optional as "metric" columns are dynamic columns in subIndex, it would be extremely inconvenient to add it to the mainIndex as well each time a new column is added. Thanks for the continued help.
          Hide
          Mikhail Khludnev added a comment -

          Parnit I asked you to use http://wiki.apache.org/solr/FunctionQuery#field ie you need to

          testServer/solr/MC_10001_CatalogEntry_en_US/select?q=

          {!scorejoin from=id to=id fromIndex=subIndex}

          metric1

          just 'metric1' nothing more, ie no need wildcard metric1:*. Note it will work if metric is number!
          if something goes wrong, add http://wiki.apache.org/solr/CommonQueryParameters#debugQuery and post here the link to the output (I hope, explain works there for scorejoin)

          regarding the inconvenience, you can declare it in the schema as dynamicField.

          Show
          Mikhail Khludnev added a comment - Parnit I asked you to use http://wiki.apache.org/solr/FunctionQuery#field ie you need to testServer/solr/MC_10001_CatalogEntry_en_US/select?q= {!scorejoin from=id to=id fromIndex=subIndex} metric1 just 'metric1' nothing more, ie no need wildcard metric1:*. Note it will work if metric is number! if something goes wrong, add http://wiki.apache.org/solr/CommonQueryParameters#debugQuery and post here the link to the output (I hope, explain works there for scorejoin) regarding the inconvenience, you can declare it in the schema as dynamicField.
          Hide
          Parnit added a comment -

          ok, trying the function query field I am getting back 0 results.
          testserver/solr/mainIndex/select?q=

          {!scorejoin%20from=id to=id fromIndex=subIndex}

          view_rank&debugQuery=true
          metric1 is also an int type like you stated

          Here is the debug
          <str name="rawquerystring">

          {!scorejoin from=id to=id fromIndex=subIndex}

          metric1
          </str>
          <str name="querystring">

          {!scorejoin from=id to=id fromIndex=subIndex}

          metric1
          </str>
          <str name="parsedquery">
          OtherCoreJoinQuery(OtherCoreJoinQuery [fromIndex=subIndex, fromCoreOpenTime=1418333501496 extends SameCoreJoinQuery [fromQuery=defaultSearch:"(metric1 view) rank", fromField=id , toField=id , scoreMode=None, boost=1.0]])
          </str>
          <str name="parsedquery_toString">
          OtherCoreJoinQuery [fromIndex=subIndex, fromCoreOpenTime=1418333501496 extends SameCoreJoinQuery [fromQuery=defaultSearch:"(metric1 view) rank", fromField=id, toField=id, scoreMode=None, boost=1.0]]
          </str>
          <lst name="explain"/>

          Show
          Parnit added a comment - ok, trying the function query field I am getting back 0 results. testserver/solr/mainIndex/select?q= {!scorejoin%20from=id to=id fromIndex=subIndex} view_rank&debugQuery=true metric1 is also an int type like you stated Here is the debug <str name="rawquerystring"> {!scorejoin from=id to=id fromIndex=subIndex} metric1 </str> <str name="querystring"> {!scorejoin from=id to=id fromIndex=subIndex} metric1 </str> <str name="parsedquery"> OtherCoreJoinQuery(OtherCoreJoinQuery [fromIndex=subIndex, fromCoreOpenTime=1418333501496 extends SameCoreJoinQuery [fromQuery=defaultSearch:"(metric1 view) rank", fromField=id , toField=id , scoreMode=None, boost=1.0] ]) </str> <str name="parsedquery_toString"> OtherCoreJoinQuery [fromIndex=subIndex, fromCoreOpenTime=1418333501496 extends SameCoreJoinQuery [fromQuery=defaultSearch:"(metric1 view) rank", fromField=id, toField=id, scoreMode=None, boost=1.0] ] </str> <lst name="explain"/>
          Hide
          Mikhail Khludnev added a comment -

          Parnit, ok...not well but, ok.
          pay attention to

          "fromQuery=defaultSearch:"(metric1 view) rank"
          

          it seems like it wasn't recognized as a function query, make a clue:

          {!func}metric1 
          

          ie

          {!scorejoin from=id to=id fromIndex=subIndex}{!func}metric1
          
          Show
          Mikhail Khludnev added a comment - Parnit , ok...not well but, ok. pay attention to "fromQuery=defaultSearch:" (metric1 view) rank" it seems like it wasn't recognized as a function query, make a clue: {!func}metric1 ie {!scorejoin from=id to=id fromIndex=subIndex}{!func}metric1
          Hide
          Parnit added a comment - - edited

          No luck unfortunately with the sorting, but now I am getting results back with just metric1 and no need for metric1:*
          using the following
          testServer/solr/mainIndex/select?q=

          {!scorejoin from=id to=id fromIndex=subIndex}{!func}metric1&debugQuery=true

          <lst name="debug">
          <str name="rawquerystring">{!scorejoin from=id to=id fromIndex=subIndex} {!func}metric1
          </str>
          <str name="querystring"> {!scorejoin from=id to=id fromIndex=subIndex}{!func}

          metric1
          </str>
          <str name="parsedquery">
          OtherCoreJoinQuery(OtherCoreJoinQuery [fromIndex=subIndex, fromCoreOpenTime=1418333501496 extends SameCoreJoinQuery [fromQuery=int(metric1), fromField=id, toField=id, scoreMode=None, boost=1.0]])
          </str>
          <str name="parsedquery_toString">
          OtherCoreJoinQuery [fromIndex=subIndex, fromCoreOpenTime=1418333501496 extends SameCoreJoinQuery [fromQuery=int(metric1), fromField=id, toField=id, scoreMode=None, boost=1.0]]
          </str>

          so just to recap metric1,metric2 return the results in the same order

          Show
          Parnit added a comment - - edited No luck unfortunately with the sorting, but now I am getting results back with just metric1 and no need for metric1:* using the following testServer/solr/mainIndex/select?q= {!scorejoin from=id to=id fromIndex=subIndex}{!func}metric1&debugQuery=true <lst name="debug"> <str name="rawquerystring">{!scorejoin from=id to=id fromIndex=subIndex} {!func}metric1 </str> <str name="querystring"> {!scorejoin from=id to=id fromIndex=subIndex}{!func} metric1 </str> <str name="parsedquery"> OtherCoreJoinQuery(OtherCoreJoinQuery [fromIndex=subIndex, fromCoreOpenTime=1418333501496 extends SameCoreJoinQuery [fromQuery=int(metric1), fromField=id, toField=id, scoreMode=None, boost=1.0] ]) </str> <str name="parsedquery_toString"> OtherCoreJoinQuery [fromIndex=subIndex, fromCoreOpenTime=1418333501496 extends SameCoreJoinQuery [fromQuery=int(metric1), fromField=id, toField=id, scoreMode=None, boost=1.0] ] </str> so just to recap metric1,metric2 return the results in the same order
          Hide
          Mikhail Khludnev added a comment -

          Parnit it clearly explain the reason scoreMode=None, I believe you just need to specify any value for this local param. I put it into description explicitly.

          Show
          Mikhail Khludnev added a comment - Parnit it clearly explain the reason scoreMode=None , I believe you just need to specify any value for this local param. I put it into description explicitly.
          Hide
          Parnit added a comment -

          I tried all 4 different scoringModes same results. Is it possible that when I introduced the metric1 column to the mainIndex we are actually scoring on that non existent field. I'm not sure how solr deals with null/empty column values. Also the scoreModes do they sort based off of document score or is it something to do with the field being passed in ie (metric1,metric2)

          Show
          Parnit added a comment - I tried all 4 different scoringModes same results. Is it possible that when I introduced the metric1 column to the mainIndex we are actually scoring on that non existent field. I'm not sure how solr deals with null/empty column values. Also the scoreModes do they sort based off of document score or is it something to do with the field being passed in ie (metric1,metric2)
          Hide
          Mikhail Khludnev added a comment -

          Parnit could you upload solr folder with index and core configs and send me the link?

          Show
          Mikhail Khludnev added a comment - Parnit could you upload solr folder with index and core configs and send me the link?
          Hide
          Parnit added a comment -

          Thanks for the help,but currently I am away from that machine and will not be access it for 3+ weeks as I will be away.
          Perhaps when I am back I could get it to you

          Show
          Parnit added a comment - Thanks for the help,but currently I am away from that machine and will not be access it for 3+ weeks as I will be away. Perhaps when I am back I could get it to you
          Hide
          Mikhail Khludnev added a comment -

          attaching fix for:

          • cross-core query parsing with test coverage.
            Note: it's the only cross-core test for the current {!join query parser! ;
          • also added multiVals into equality&hash
          Show
          Mikhail Khludnev added a comment - attaching fix for: cross-core query parsing with test coverage. Note: it's the only cross-core test for the current {!join query parser! ; also added multiVals into equality&hash
          Hide
          Mikhail Khludnev added a comment -

          that's would be great if we replicate SOLR-4905 here also. Please vote, let me know if you need it.

          Show
          Mikhail Khludnev added a comment - that's would be great if we replicate SOLR-4905 here also. Please vote, let me know if you need it.
          Hide
          Ryan Josal added a comment -

          This is awesome, the normal !join qparser is mainly only good for fq, but with scoring, this is good as a q or subquery.

          Show
          Ryan Josal added a comment - This is awesome, the normal !join qparser is mainly only good for fq, but with scoring, this is good as a q or subquery.
          Hide
          Neeraj Lajpal added a comment -

          I have a doubt regarding query time join.
          Example:
          I have the document structure like this:
          parent1: id, important_category
          child1: id, root,brandid
          child2:id,root,brandid

          root field of child doc contains value of parent's id.

          I want to use query time join. I want all the parent documents after applying boosts on children's fields and avg their scores, and then apply boost on important_category field on parent.
          My query is like this without boost on parent's field:

          {!scorejoin from=_root_ to=id score=avg multiVals=false}(brandid:1398^4 OR brandid:237^4.5)
          It is working fine.

          But, I want to apply boost to important_category field of parent also.
          If I will use fq it will not impact score, so I have to use q only.
          If I make q like : important_category:322^4{!scorejoin from=_root_ to=id score=avg multiVals=false}

          (brandid:1398^4 OR brandid:237^4.5)
          it is not showing any results.
          How can I do this?

          Show
          Neeraj Lajpal added a comment - I have a doubt regarding query time join. Example: I have the document structure like this: parent1: id, important_category child1: id, root ,brandid child2:id, root ,brandid root field of child doc contains value of parent's id. I want to use query time join. I want all the parent documents after applying boosts on children's fields and avg their scores, and then apply boost on important_category field on parent. My query is like this without boost on parent's field: {!scorejoin from=_root_ to=id score=avg multiVals=false}(brandid:1398^4 OR brandid:237^4.5) It is working fine. But, I want to apply boost to important_category field of parent also. If I will use fq it will not impact score, so I have to use q only. If I make q like : important_category:322^4{!scorejoin from=_root_ to=id score=avg multiVals=false} (brandid:1398^4 OR brandid:237^4.5) it is not showing any results. How can I do this?
          Hide
          Ryan Josal added a comment - - edited

          Neeraj Lajpal Solr parses the value of q with a single parser, which will be whatever your defType is. In order to do something like that you can use nested queries:

          important_category:322^4 _query_:{!scorejoin from=_root_ to=id score=avg multiVals=false v=$jq}&jq=(brandid:1398^4 OR brandid:237^4.5)
          
          Show
          Ryan Josal added a comment - - edited Neeraj Lajpal Solr parses the value of q with a single parser, which will be whatever your defType is. In order to do something like that you can use nested queries: important_category:322^4 _query_:{!scorejoin from=_root_ to=id score=avg multiVals=false v=$jq}&jq=(brandid:1398^4 OR brandid:237^4.5)
          Hide
          Neeraj Lajpal added a comment -

          Thanks a lot Ryan.
          This solved my problem.
          But, needed to change a couple of things, enclose the value of query in double quotes and remove jq.
          Without these changes it was not showing any result.

          I executed this query:
          important_category:322^4 query:"

          {!scorejoin from=_root_ to=id score=avg multiVals=false}(brandid:1398^4 OR brandid:237^4.5)"

          "debug": {
          "rawquerystring": "important_cat:322^4 query:\"{!scorejoin from=_root_ to=id score=avg multiVals=false}

          (brandid:1398^4 OR brandid:237^4.5)\"",
          "querystring": "important_cat:322^4 query:\"

          {!scorejoin from=_root_ to=id score=avg multiVals=false}

          (brandid:1398^4 OR brandid:237^4.5)\"",
          "parsedquery": "important_cat:322^4.0 SameCoreJoinQuery(SameCoreJoinQuery [fromQuery=brandid:1398^4.0 brandid:237^4.5, fromField=_root_, toField=id, scoreMode=Avg, boost=1.0, multiVals=false])"
          }

          Everything looks fine.
          Thanks again.

          Show
          Neeraj Lajpal added a comment - Thanks a lot Ryan. This solved my problem. But, needed to change a couple of things, enclose the value of query in double quotes and remove jq. Without these changes it was not showing any result. I executed this query: important_category:322^4 query :" {!scorejoin from=_root_ to=id score=avg multiVals=false}(brandid:1398^4 OR brandid:237^4.5)" "debug": { "rawquerystring": "important_cat:322^4 query :\"{!scorejoin from=_root_ to=id score=avg multiVals=false} (brandid:1398^4 OR brandid:237^4.5)\"", "querystring": "important_cat:322^4 query :\" {!scorejoin from=_root_ to=id score=avg multiVals=false} (brandid:1398^4 OR brandid:237^4.5)\"", "parsedquery": "important_cat:322^4.0 SameCoreJoinQuery(SameCoreJoinQuery [fromQuery=brandid:1398^4.0 brandid:237^4.5, fromField=_root_, toField=id, scoreMode=Avg, boost=1.0, multiVals=false] )" } Everything looks fine. Thanks again.
          Hide
          Neeraj Lajpal added a comment -

          Thanks a lot Ryan.
          This solved my problem.
          But, needed to change a couple of things, enclose the value of query in double quotes and remove jq.
          Without these changes it was not showing any result.

          I executed this query:
          important_category:322^4 query:"

          {!scorejoin from=_root_ to=id score=avg multiVals=false}(brandid:1398^4 OR brandid:237^4.5)"

          "debug": {
          "rawquerystring": "important_cat:322^4 query:\"{!scorejoin from=_root_ to=id score=avg multiVals=false}

          (brandid:1398^4 OR brandid:237^4.5)\"",
          "querystring": "important_cat:322^4 query:\"

          {!scorejoin from=_root_ to=id score=avg multiVals=false}

          (brandid:1398^4 OR brandid:237^4.5)\"",
          "parsedquery": "important_cat:322^4.0 SameCoreJoinQuery(SameCoreJoinQuery [fromQuery=brandid:1398^4.0 brandid:237^4.5, fromField=_root_, toField=id, scoreMode=Avg, boost=1.0, multiVals=false])"
          }

          Everything looks fine.
          Thanks again.

          Show
          Neeraj Lajpal added a comment - Thanks a lot Ryan. This solved my problem. But, needed to change a couple of things, enclose the value of query in double quotes and remove jq. Without these changes it was not showing any result. I executed this query: important_category:322^4 query :" {!scorejoin from=_root_ to=id score=avg multiVals=false}(brandid:1398^4 OR brandid:237^4.5)" "debug": { "rawquerystring": "important_cat:322^4 query :\"{!scorejoin from=_root_ to=id score=avg multiVals=false} (brandid:1398^4 OR brandid:237^4.5)\"", "querystring": "important_cat:322^4 query :\" {!scorejoin from=_root_ to=id score=avg multiVals=false} (brandid:1398^4 OR brandid:237^4.5)\"", "parsedquery": "important_cat:322^4.0 SameCoreJoinQuery(SameCoreJoinQuery [fromQuery=brandid:1398^4.0 brandid:237^4.5, fromField=_root_, toField=id, scoreMode=Avg, boost=1.0, multiVals=false] )" } Everything looks fine. Thanks again.
          Hide
          Ryan Josal added a comment -

          I've been using this for a few days now, and here are my comments:
          *) it's more functionally useful to have the scores returned with this than the !join parser.
          *) basing it off JoinUtil is a clean solution that centralizes this type of logic.
          *) That being said, scorejoin has performed worse than !join in my tests, usually almost twice as slow. It's not an exhaustive test by any means, just an observation. So it doesn't seem fit to replace !join as it is, although it does feel like this is the right direction to be going for joins.
          *) JoinUtil does not currently support multivalue fields for the "to" index. It chooses one of the matching values instead of applying the ScoreMode for all matching documents. It's a fairly small change to TermsIncludingScoreQuery to add this feature.

          Show
          Ryan Josal added a comment - I've been using this for a few days now, and here are my comments: *) it's more functionally useful to have the scores returned with this than the !join parser. *) basing it off JoinUtil is a clean solution that centralizes this type of logic. *) That being said, scorejoin has performed worse than !join in my tests, usually almost twice as slow. It's not an exhaustive test by any means, just an observation. So it doesn't seem fit to replace !join as it is, although it does feel like this is the right direction to be going for joins. *) JoinUtil does not currently support multivalue fields for the "to" index. It chooses one of the matching values instead of applying the ScoreMode for all matching documents. It's a fairly small change to TermsIncludingScoreQuery to add this feature.
          Hide
          Ryan Josal added a comment -

          Neeraj Lajpal, glad you got it working. There might be a caution with using !scorejoin this way. The scores coming back from !scorejoin are from the query to the fromIndex, and so they're not really related to the score coming from the important_category matches. The matches from the scorejoin might have a really high max score, while the matches from the important_category might have a really low max_score, in a relative sense. Or vice versa. I think this use case is workable but it's something to keep in mind.

          Show
          Ryan Josal added a comment - Neeraj Lajpal , glad you got it working. There might be a caution with using !scorejoin this way. The scores coming back from !scorejoin are from the query to the fromIndex, and so they're not really related to the score coming from the important_category matches. The matches from the scorejoin might have a really high max score, while the matches from the important_category might have a really low max_score, in a relative sense. Or vice versa. I think this use case is workable but it's something to keep in mind.
          Hide
          Neeraj Lajpal added a comment -

          Thanks for pointing that out Ryan.
          I will keep that in mind while designing algo that will decide boosting params.

          Show
          Neeraj Lajpal added a comment - Thanks for pointing that out Ryan. I will keep that in mind while designing algo that will decide boosting params.
          Hide
          Timothy Potter added a comment -

          I'm working to get this patch committed to trunk / 5x

          Show
          Timothy Potter added a comment - I'm working to get this patch committed to trunk / 5x
          Hide
          Mikhail Khludnev added a comment -

          Tell me if I need to improve something! Thanks!

          Show
          Mikhail Khludnev added a comment - Tell me if I need to improve something! Thanks!
          Hide
          Ryan Josal added a comment -

          That will be great Tim! For my personal use case, I added a feature to the qparser that, after constructing the LocalSolrQueryRequest, would apply the handler config (defaults, appends, invariants) from the otherCore (say "/select" handler) to the request so that in a nutshell your other query could be a simple string and it would follow whatever edismax rules you had configured for the other core. That instead of the more strict db style lucene query syntax. If there's interest I am happy to share the patch.

          Show
          Ryan Josal added a comment - That will be great Tim! For my personal use case, I added a feature to the qparser that, after constructing the LocalSolrQueryRequest, would apply the handler config (defaults, appends, invariants) from the otherCore (say "/select" handler) to the request so that in a nutshell your other query could be a simple string and it would follow whatever edismax rules you had configured for the other core. That instead of the more strict db style lucene query syntax. If there's interest I am happy to share the patch.
          Hide
          Timothy Potter added a comment -

          Ryan Josal sounds cool ... definitely share the patch!

          Show
          Timothy Potter added a comment - Ryan Josal sounds cool ... definitely share the patch!
          Hide
          Ryan Josal added a comment - - edited

          I've attached otherHandler.patch, which is a patch on top of the existing ScoreJoinQParserPlugin.java from the Dec 14 patch, which shows changes to add a feature where the user can supply handler=/select (for example) in localparams, to apply the config from the handler in the other core. You'll notice that the request is built from localParams instead of "params" so that whatever your current core params are don't override the other core configured params. If this breaks something, it could be changed to only do that when the "handler" localParam is present. This patch is quick and dirty because it uses reflection to get the defaults,appends,invariants config from the BaseRequestHandler class. Really, the access level of those variables should change, or the method should be moved there.

          Personally I'm using this feature to join my deals core with my products core as the othercore, and I wanted it to do a regular search against the products core. This approach could actually be used in the !join qparser too. If this patch is useful for somebody, great!

          Show
          Ryan Josal added a comment - - edited I've attached otherHandler.patch, which is a patch on top of the existing ScoreJoinQParserPlugin.java from the Dec 14 patch, which shows changes to add a feature where the user can supply handler=/select (for example) in localparams, to apply the config from the handler in the other core. You'll notice that the request is built from localParams instead of "params" so that whatever your current core params are don't override the other core configured params. If this breaks something, it could be changed to only do that when the "handler" localParam is present. This patch is quick and dirty because it uses reflection to get the defaults,appends,invariants config from the BaseRequestHandler class. Really, the access level of those variables should change, or the method should be moved there. Personally I'm using this feature to join my deals core with my products core as the othercore, and I wanted it to do a regular search against the products core. This approach could actually be used in the !join qparser too. If this patch is useful for somebody, great!
          Hide
          Timothy Potter added a comment -

          Here's an updated patch for trunk. The JavaDoc on the ScoreJoinQParserPlugin wasn't passing precommit, so I need to fix that before committing (removed it for now).

          This does not have the code from the patch Ryan posted yet ...

          Mikhail Khludnev You mentioned this solution would address SOLR-6357, but I'm still seeing the same error when using scorejoin. Can you elaborate on why you think this fixes that problem?

          [~/dev/lw/projects/solr_trunk_co/solr]$ curl "http://localhost:8983/solr/techproducts/update?commit=true" -d '<delete><query>{!scorejoin from=manu_id_s to=id}ipod</query></delete>'
          <?xml version="1.0" encoding="UTF-8"?>
          <response>
          <lst name="responseHeader"><int name="status">500</int><int name="QTime">6</int></lst><lst name="error"><str name="msg">org.apache.lucene.search.IndexSearcher cannot be cast to org.apache.solr.search.SolrIndexSearcher</str><str name="trace">java.lang.ClassCastException: org.apache.lucene.search.IndexSearcher cannot be cast to org.apache.solr.search.SolrIndexSearcher
          	at org.apache.solr.search.JoinQuery.createWeight(JoinQParserPlugin.java:207)
          	at org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:704)
          	at org.apache.lucene.search.BooleanWeight.&lt;init&gt;(BooleanWeight.java:56)
          	at org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:197)
          	at org.apache.solr.update.DeleteByQueryWrapper.createWeight(DeleteByQueryWrapper.java:72)
          	at org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:704)
          	at org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:687)
          	at org.apache.lucene.search.QueryWrapperFilter.getDocIdSet(QueryWrapperFilter.java:63)
          	at org.apache.lucene.index.BufferedUpdatesStream.applyQueryDeletes(BufferedUpdatesStream.java:690)
          	at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:261)
          	at org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3148)
          	at org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3134)
          	at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2808)
          	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2962)
          	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2929)
          
          Show
          Timothy Potter added a comment - Here's an updated patch for trunk. The JavaDoc on the ScoreJoinQParserPlugin wasn't passing precommit, so I need to fix that before committing (removed it for now). This does not have the code from the patch Ryan posted yet ... Mikhail Khludnev You mentioned this solution would address SOLR-6357 , but I'm still seeing the same error when using scorejoin. Can you elaborate on why you think this fixes that problem? [~/dev/lw/projects/solr_trunk_co/solr]$ curl "http: //localhost:8983/solr/techproducts/update?commit= true " -d '<delete><query>{!scorejoin from=manu_id_s to=id}ipod</query></delete>' <?xml version= "1.0" encoding= "UTF-8" ?> <response> <lst name= "responseHeader" >< int name= "status" >500</ int >< int name= "QTime" >6</ int ></lst><lst name= "error" ><str name= "msg" >org.apache.lucene.search.IndexSearcher cannot be cast to org.apache.solr.search.SolrIndexSearcher</str><str name= "trace" >java.lang.ClassCastException: org.apache.lucene.search.IndexSearcher cannot be cast to org.apache.solr.search.SolrIndexSearcher at org.apache.solr.search.JoinQuery.createWeight(JoinQParserPlugin.java:207) at org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:704) at org.apache.lucene.search.BooleanWeight.&lt;init&gt;(BooleanWeight.java:56) at org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:197) at org.apache.solr.update.DeleteByQueryWrapper.createWeight(DeleteByQueryWrapper.java:72) at org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:704) at org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:687) at org.apache.lucene.search.QueryWrapperFilter.getDocIdSet(QueryWrapperFilter.java:63) at org.apache.lucene.index.BufferedUpdatesStream.applyQueryDeletes(BufferedUpdatesStream.java:690) at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:261) at org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3148) at org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3134) at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2808) at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2962) at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2929)
          Hide
          Timothy Potter added a comment -

          Actually ... hold that thought ... looks like code is still hitting JoinQParserPlugin.java for some reason ??? I'll dig.

          Show
          Timothy Potter added a comment - Actually ... hold that thought ... looks like code is still hitting JoinQParserPlugin.java for some reason ??? I'll dig.
          Hide
          Timothy Potter added a comment -

          Well, nm ... I must not have done a full clean / rebuild. After doing so, it is working! Sorry for the noise ... will start looking into Ryan's contribution as well now. Thanks Mikhail Khludnev - great feature!

          Show
          Timothy Potter added a comment - Well, nm ... I must not have done a full clean / rebuild. After doing so, it is working! Sorry for the noise ... will start looking into Ryan's contribution as well now. Thanks Mikhail Khludnev - great feature!
          Hide
          Timothy Potter added a comment -

          Mikhail Khludnev I'm wondering if we should add the solution for SOLR-4905 to the scorejoin parser too? It seems like this would be a good thing for scorejoin to support as well.

          Show
          Timothy Potter added a comment - Mikhail Khludnev I'm wondering if we should add the solution for SOLR-4905 to the scorejoin parser too? It seems like this would be a good thing for scorejoin to support as well.
          Hide
          David Smiley added a comment -

          One tiny bit of input I have on this is rather cosmetic but nonetheless it'd be nice if the scoring mode could be specified via a local-param on the existing {!join} QParserPlugin. It's still a join, we just want the score too. It very well may have a different implementation when it's scoring but that's an implementation detail.

          Show
          David Smiley added a comment - One tiny bit of input I have on this is rather cosmetic but nonetheless it'd be nice if the scoring mode could be specified via a local-param on the existing {!join} QParserPlugin. It's still a join, we just want the score too. It very well may have a different implementation when it's scoring but that's an implementation detail.
          Hide
          Ryan Josal added a comment -

          That makes a lot of sense David, I would prefer this as it would reduce confusion, some similar code, and make implementation changes in the future more flexible.

          Show
          Ryan Josal added a comment - That makes a lot of sense David, I would prefer this as it would reduce confusion, some similar code, and make implementation changes in the future more flexible.
          Hide
          Mikhail Khludnev added a comment -

          Timothy Potter I like SOLR-4905 much. If it's possible I prefer to create a patch right after SOLR-6234 is committed, just prefer to work on smaller units. Let me know if you see it as a blocker, and I'll dig into SOLR-4905 (I still unaware of its' details).

          David Smiley I'm neutral about this - I agree that it would be convenient, but have a concern, nevermind.
          Here is another off-top-just-a-thought: since Lucene has query/filter cache {!join} algorithm can be migrated on intrinsic Lucene's IndexSearcher.

          Show
          Mikhail Khludnev added a comment - Timothy Potter I like SOLR-4905 much. If it's possible I prefer to create a patch right after SOLR-6234 is committed, just prefer to work on smaller units. Let me know if you see it as a blocker, and I'll dig into SOLR-4905 (I still unaware of its' details). David Smiley I'm neutral about this - I agree that it would be convenient, but have a concern, nevermind. Here is another off-top-just-a-thought: since Lucene has query/filter cache {!join} algorithm can be migrated on intrinsic Lucene's IndexSearcher.
          Hide
          Timothy Potter added a comment -

          Ok, we can tackle adding 4905 as a separate ticket.

          Regarding David's comment, what does

          {!join} do that {!scorejoin} does not? (besides 4905) if nothing, then it seems to make sense to just replace {!join}

          with this implementation vs. introducing a new qparser type.

          Show
          Timothy Potter added a comment - Ok, we can tackle adding 4905 as a separate ticket. Regarding David's comment, what does {!join} do that {!scorejoin} does not? (besides 4905) if nothing, then it seems to make sense to just replace {!join} with this implementation vs. introducing a new qparser type.
          Hide
          Yonik Seeley added a comment -

          Regarding David's comment, what does {!join} do that {!scorejoin} does not? (besides 4905) if nothing, then it seems to make sense to just replace {!join}

          From an API perspective, nothing that I'm aware of. From an implementation perspective, it can be faster or slower depending on the details (it's a different implementation). So it makes sense to perhaps consolidate the APIs, but keep the different implementations (sort of like facet.method does for faceting).

          Show
          Yonik Seeley added a comment - Regarding David's comment, what does {!join} do that {!scorejoin} does not? (besides 4905) if nothing, then it seems to make sense to just replace {!join} From an API perspective, nothing that I'm aware of. From an implementation perspective, it can be faster or slower depending on the details (it's a different implementation). So it makes sense to perhaps consolidate the APIs, but keep the different implementations (sort of like facet.method does for faceting).
          Hide
          Timothy Potter added a comment -

          Thanks Yonik Seeley, makes sense.

          Mikhail Khludnev I didn't quite grok your last comment:

          I'm neutral about this - I agree that it would be convenient, but have a concern, nevermind. Here is another off-top-just-a-thought: since Lucene has query/filter cache {!join} algorithm can be migrated on intrinsic Lucene's IndexSearcher.

          Can you elaborate?

          Show
          Timothy Potter added a comment - Thanks Yonik Seeley , makes sense. Mikhail Khludnev I didn't quite grok your last comment: I'm neutral about this - I agree that it would be convenient, but have a concern, nevermind. Here is another off-top-just-a-thought: since Lucene has query/filter cache {!join} algorithm can be migrated on intrinsic Lucene's IndexSearcher. Can you elaborate?
          Hide
          Yonik Seeley added a comment - - edited

          Related: my last few commits on heliosearch related to refactoring Solr's Join stuff to accommodate multiple implementations:
          https://github.com/Heliosearch/heliosearch/commits/helio_4_10

          In the short term though, it may be easiest for JoinQParser to detect "score=..." and delegate to this new qparser?

          I haven't had a chance to look at this patch... from an API perspective, can someone explain what the "multiVals=true|false" parameter is supposed to do?
          edit: Oh, and this: "supports b=100 param to pass Query.setBoost()." - what is the effect of this? (exactly what query is the boost applied to and why aren't we using standard boosting mechanisms?)

          Show
          Yonik Seeley added a comment - - edited Related: my last few commits on heliosearch related to refactoring Solr's Join stuff to accommodate multiple implementations: https://github.com/Heliosearch/heliosearch/commits/helio_4_10 In the short term though, it may be easiest for JoinQParser to detect "score=..." and delegate to this new qparser? I haven't had a chance to look at this patch... from an API perspective, can someone explain what the "multiVals=true|false" parameter is supposed to do? edit: Oh, and this: "supports b=100 param to pass Query.setBoost()." - what is the effect of this? (exactly what query is the boost applied to and why aren't we using standard boosting mechanisms?)
          Hide
          Mikhail Khludnev added a comment - - edited

          it may be easiest for JoinQParser to detect "score=..." and delegate

          +1

          the "multiVals=true|false" parameter is supposed to do?

          it's just exposing JoinUtil.createJoinQuery() functionality, it implies that fromField is multivalue SortedSetDV and these values are looped.
          another interesting feature request which I've heard is to treat toField multivalue and hence accumulate several from-side scores in to-side values. it's not in the patch, and should be done in JoinUtil first.

          exactly what query is the boost applied to

          this parser calls JoinUtil.createJoinQuery() and then calls setBoost() on its' result.

          why aren't we using standard boosting mechanisms?

          I might be unaware of 'standard boosting mechanisms' I only aware about {!boost but people ask for more sugar.

          Show
          Mikhail Khludnev added a comment - - edited it may be easiest for JoinQParser to detect "score=..." and delegate +1 the "multiVals=true|false" parameter is supposed to do? it's just exposing JoinUtil.createJoinQuery() functionality, it implies that fromField is multivalue SortedSetDV and these values are looped. another interesting feature request which I've heard is to treat toField multivalue and hence accumulate several from-side scores in to-side values. it's not in the patch, and should be done in JoinUtil first. exactly what query is the boost applied to this parser calls JoinUtil.createJoinQuery() and then calls setBoost() on its' result. why aren't we using standard boosting mechanisms? I might be unaware of 'standard boosting mechanisms' I only aware about {!boost but people ask for more sugar.
          Hide
          Mikhail Khludnev added a comment -

          I think it's not so important.

          Show
          Mikhail Khludnev added a comment - I think it's not so important.
          Hide
          Ryan Josal added a comment - - edited

          It seems like the parser can find out if it's multivalued for itself by checking the otherCore's Schema.

          I think multivalued "to" field on the current core should be a separate ticket since it is not a feature of the qparser, it's an improvement to JoinUtil. I can have a patch for that later; I have it working.

          Show
          Ryan Josal added a comment - - edited It seems like the parser can find out if it's multivalued for itself by checking the otherCore's Schema. I think multivalued "to" field on the current core should be a separate ticket since it is not a feature of the qparser, it's an improvement to JoinUtil. I can have a patch for that later; I have it working.
          Hide
          Yonik Seeley added a comment -

          it's just exposing JoinUtil.createJoinQuery() functionality, it implies that fromField is multivalue SortedSetDV

          That's a Java API that can change from release-to-release. Our HTTP APIs should stand alone and not be defined in terms of implementation details.
          In this specific case, we should either know or be able to figure that out, right? The user shouldn't have to specify.

          this parser calls JoinUtil.createJoinQuery() and then calls setBoost() on its' result.

          Seems like that should be a separate issue (boosting the current query from within local params for that query). If it's a good idea, it should be done in a generic way for all queries / qparsers.

          Show
          Yonik Seeley added a comment - it's just exposing JoinUtil.createJoinQuery() functionality, it implies that fromField is multivalue SortedSetDV That's a Java API that can change from release-to-release. Our HTTP APIs should stand alone and not be defined in terms of implementation details. In this specific case, we should either know or be able to figure that out, right? The user shouldn't have to specify. this parser calls JoinUtil.createJoinQuery() and then calls setBoost() on its' result. Seems like that should be a separate issue (boosting the current query from within local params for that query). If it's a good idea, it should be done in a generic way for all queries / qparsers.
          Hide
          Timothy Potter added a comment -

          Mikhail Khludnev are you going to update the latest patch I posted to have

          JoinQParser to detect "score=..." and delegate

          Show
          Timothy Potter added a comment - Mikhail Khludnev are you going to update the latest patch I posted to have JoinQParser to detect "score=..." and delegate
          Hide
          Mikhail Khludnev added a comment -

          I can, if you wish, as well as handle other suggestions, but only in the end of this week.

          Show
          Mikhail Khludnev added a comment - I can, if you wish, as well as handle other suggestions, but only in the end of this week.
          Hide
          Mikhail Khludnev added a comment -

          ok! here we are:

          • qparser is called from the older one (JoinQParserPlugin) by specifying score local param {join score=...}
          • b= is removed in favor of SOLR-7814
          • multiVals was just removed too, it seems docvalues hide multivalued when singlevalued is requested
            want to watch?
          Show
          Mikhail Khludnev added a comment - ok! here we are: qparser is called from the older one (JoinQParserPlugin) by specifying score local param {join score=...} b= is removed in favor of SOLR-7814 multiVals was just removed too, it seems docvalues hide multivalued when singlevalued is requested want to watch?
          Hide
          Timothy Potter added a comment -

          looks good Mikhail Khludnev +1 to commit. Please be sure to add documentation for this new feature to the refguide. I'll add a separate unit test that uses this feature to verify SOLR-6357 once this is committed.

          Show
          Timothy Potter added a comment - looks good Mikhail Khludnev +1 to commit. Please be sure to add documentation for this new feature to the refguide. I'll add a separate unit test that uses this feature to verify SOLR-6357 once this is committed.
          Hide
          Mikhail Khludnev added a comment -

          added javadocs and changes. I'm going to commit it tomorrow. Is there any objections? Shouldn't it be merged into 5x right there?

          Show
          Mikhail Khludnev added a comment - added javadocs and changes. I'm going to commit it tomorrow. Is there any objections? Shouldn't it be merged into 5x right there?
          Hide
          Mikhail Khludnev added a comment -

          found bug in toString() added test coverage.

          Show
          Mikhail Khludnev added a comment - found bug in toString() added test coverage.
          Hide
          Mikhail Khludnev added a comment -

          added link to lucene join for javadocs.

          Show
          Mikhail Khludnev added a comment - added link to lucene join for javadocs.
          Hide
          ASF subversion and git services added a comment -

          Commit 1693092 from mkhl@apache.org in branch 'dev/trunk'
          [ https://svn.apache.org/r1693092 ]

          SOLR-6234: Scoring for query time join

          Show
          ASF subversion and git services added a comment - Commit 1693092 from mkhl@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1693092 ] SOLR-6234 : Scoring for query time join
          Hide
          ASF subversion and git services added a comment -

          Commit 1693101 from mkhl@apache.org in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1693101 ]

          SOLR-6234: Scoring for query time join

          Show
          ASF subversion and git services added a comment - Commit 1693101 from mkhl@apache.org in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1693101 ] SOLR-6234 : Scoring for query time join
          Hide
          Timothy Potter added a comment -

          Looks like some problems with the Javadoc still ... I'm getting this from running the smoke tester. Mikhail Khludnev did you run precommit?

             [smoker] file:///Users/timpotter/dev/lw/projects/solr_trunk_co/lucene/build/smokeTestRelease/tmp/unpack/solr-6.0.0/solr/build/docs/solr-core/org/apache/solr/search/join/ScoreJoinQParserPlugin.html
             [smoker]   BROKEN LINK: file:///Users/timpotter/dev/lw/projects/solr_trunk_co/lucene/build/smokeTestRelease/tmp/unpack/solr-6.0.0/lucene/build/docs/core/org/apache/lucene/search/join.ScoreMode.html
             [smoker] Traceback (most recent call last):
             [smoker]   File "/Users/timpotter/dev/lw/projects/solr_trunk_co/dev-tools/scripts/smokeTestRelease.py", line 1463, in <module>
             [smoker]     main()
             [smoker]   File "/Users/timpotter/dev/lw/projects/solr_trunk_co/dev-tools/scripts/smokeTestRelease.py", line 1408, in main
             [smoker]     smokeTest(c.java, c.url, c.revision, c.version, c.tmp_dir, c.is_signed, ' '.join(c.test_args))
             [smoker]   File "/Users/timpotter/dev/lw/projects/solr_trunk_co/dev-tools/scripts/smokeTestRelease.py", line 1453, in smokeTest
             [smoker]     unpackAndVerify(java, 'solr', tmpDir, 'solr-%s-src.tgz' % version, svnRevision, version, testArgs, baseURL)
             [smoker]   File "/Users/timpotter/dev/lw/projects/solr_trunk_co/dev-tools/scripts/smokeTestRelease.py", line 592, in unpackAndVerify
             [smoker]     verifyUnpacked(java, project, artifact, unpackPath, svnRevision, version, testArgs, tmpDir, baseURL)
             [smoker]   File "/Users/timpotter/dev/lw/projects/solr_trunk_co/dev-tools/scripts/smokeTestRelease.py", line 701, in verifyUnpacked
             [smoker]     checkJavadocpathFull('%s/solr/build/docs' % unpackPath, False)
             [smoker]   File "/Users/timpotter/dev/lw/projects/solr_trunk_co/dev-tools/scripts/smokeTestRelease.py", line 893, in checkJavadocpathFull
             [smoker]     raise RuntimeError('broken javadocs links found!')
             [smoker] RuntimeError: broken javadocs links found!
          
          Show
          Timothy Potter added a comment - Looks like some problems with the Javadoc still ... I'm getting this from running the smoke tester. Mikhail Khludnev did you run precommit? [smoker] file: ///Users/timpotter/dev/lw/projects/solr_trunk_co/lucene/build/smokeTestRelease/tmp/unpack/solr-6.0.0/solr/build/docs/solr-core/org/apache/solr/search/join/ScoreJoinQParserPlugin.html [smoker] BROKEN LINK: file: ///Users/timpotter/dev/lw/projects/solr_trunk_co/lucene/build/smokeTestRelease/tmp/unpack/solr-6.0.0/lucene/build/docs/core/org/apache/lucene/search/join.ScoreMode.html [smoker] Traceback (most recent call last): [smoker] File "/Users/timpotter/dev/lw/projects/solr_trunk_co/dev-tools/scripts/smokeTestRelease.py" , line 1463, in <module> [smoker] main() [smoker] File "/Users/timpotter/dev/lw/projects/solr_trunk_co/dev-tools/scripts/smokeTestRelease.py" , line 1408, in main [smoker] smokeTest(c.java, c.url, c.revision, c.version, c.tmp_dir, c.is_signed, ' '.join(c.test_args)) [smoker] File "/Users/timpotter/dev/lw/projects/solr_trunk_co/dev-tools/scripts/smokeTestRelease.py" , line 1453, in smokeTest [smoker] unpackAndVerify(java, 'solr', tmpDir, 'solr-%s-src.tgz' % version, svnRevision, version, testArgs, baseURL) [smoker] File "/Users/timpotter/dev/lw/projects/solr_trunk_co/dev-tools/scripts/smokeTestRelease.py" , line 592, in unpackAndVerify [smoker] verifyUnpacked(java, project, artifact, unpackPath, svnRevision, version, testArgs, tmpDir, baseURL) [smoker] File "/Users/timpotter/dev/lw/projects/solr_trunk_co/dev-tools/scripts/smokeTestRelease.py" , line 701, in verifyUnpacked [smoker] checkJavadocpathFull('%s/solr/build/docs' % unpackPath, False) [smoker] File "/Users/timpotter/dev/lw/projects/solr_trunk_co/dev-tools/scripts/smokeTestRelease.py" , line 893, in checkJavadocpathFull [smoker] raise RuntimeError('broken javadocs links found!') [smoker] RuntimeError: broken javadocs links found!
          Hide
          Mikhail Khludnev added a comment -

          the latest patch adds a tag into solr/common-build.xml which fixes it. I did run precommit for sure a plenty of times. fwiw build passed https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java8/245/

          Show
          Mikhail Khludnev added a comment - the latest patch adds a tag into solr/common-build.xml which fixes it. I did run precommit for sure a plenty of times. fwiw build passed https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java8/245/
          Hide
          Uwe Schindler added a comment - - edited

          The problem still occurs in nightly's smoker builds (both 5.x and trunk). Was this committed?

             [smoker] Verify...
             [smoker] 
             [smoker] file:///x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/lucene/build/smokeTestRelease/tmp/unpack/solr-5.3.0/solr/build/docs/solr-core/org/apache/solr/search/join/ScoreJoinQParserPlugin.html
             [smoker]   BROKEN LINK: file:///x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/lucene/build/smokeTestRelease/tmp/unpack/solr-5.3.0/lucene/build/docs/core/org/apache/lucene/search/join.ScoreMode.html
             [smoker] Traceback (most recent call last):
             [smoker]   File "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/dev-tools/scripts/smokeTestRelease.py", line 1487, in <module>
             [smoker]     main()
             [smoker]   File "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/dev-tools/scripts/smokeTestRelease.py", line 1432, in main
             [smoker]     smokeTest(c.java, c.url, c.revision, c.version, c.tmp_dir, c.is_signed, ' '.join(c.test_args))
             [smoker]   File "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/dev-tools/scripts/smokeTestRelease.py", line 1477, in smokeTest
             [smoker]     unpackAndVerify(java, 'solr', tmpDir, 'solr-%s-src.tgz' % version, svnRevision, version, testArgs, baseURL)
             [smoker]   File "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/dev-tools/scripts/smokeTestRelease.py", line 583, in unpackAndVerify
             [smoker]     verifyUnpacked(java, project, artifact, unpackPath, svnRevision, version, testArgs, tmpDir, baseURL)
             [smoker]   File "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/dev-tools/scripts/smokeTestRelease.py", line 702, in verifyUnpacked
             [smoker]     checkJavadocpathFull('%s/solr/build/docs' % unpackPath, False)
             [smoker]   File "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/dev-tools/scripts/smokeTestRelease.py", line 918, in checkJavadocpathFull
             [smoker]     raise RuntimeError('broken javadocs links found!')
             [smoker] RuntimeError: broken javadocs links found!
          

          The link is definitely wrong as it goes to core module not join.

          Show
          Uwe Schindler added a comment - - edited The problem still occurs in nightly's smoker builds (both 5.x and trunk). Was this committed? [smoker] Verify... [smoker] [smoker] file:///x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/lucene/build/smokeTestRelease/tmp/unpack/solr-5.3.0/solr/build/docs/solr-core/org/apache/solr/search/join/ScoreJoinQParserPlugin.html [smoker] BROKEN LINK: file:///x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/lucene/build/smokeTestRelease/tmp/unpack/solr-5.3.0/lucene/build/docs/core/org/apache/lucene/search/join.ScoreMode.html [smoker] Traceback (most recent call last): [smoker] File "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/dev-tools/scripts/smokeTestRelease.py", line 1487, in <module> [smoker] main() [smoker] File "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/dev-tools/scripts/smokeTestRelease.py", line 1432, in main [smoker] smokeTest(c.java, c.url, c.revision, c.version, c.tmp_dir, c.is_signed, ' '.join(c.test_args)) [smoker] File "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/dev-tools/scripts/smokeTestRelease.py", line 1477, in smokeTest [smoker] unpackAndVerify(java, 'solr', tmpDir, 'solr-%s-src.tgz' % version, svnRevision, version, testArgs, baseURL) [smoker] File "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/dev-tools/scripts/smokeTestRelease.py", line 583, in unpackAndVerify [smoker] verifyUnpacked(java, project, artifact, unpackPath, svnRevision, version, testArgs, tmpDir, baseURL) [smoker] File "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/dev-tools/scripts/smokeTestRelease.py", line 702, in verifyUnpacked [smoker] checkJavadocpathFull('%s/solr/build/docs' % unpackPath, False) [smoker] File "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-5.x/dev-tools/scripts/smokeTestRelease.py", line 918, in checkJavadocpathFull [smoker] raise RuntimeError('broken javadocs links found!') [smoker] RuntimeError: broken javadocs links found! The link is definitely wrong as it goes to core module not join.
          Hide
          Adrien Grand added a comment -

          It might be related to the fact that ScoreJoinQParserPlugin was put in a different package in 5.x compared to trunk? (my IDE complains that the package declaration does not match the folder the class is in)

          Show
          Adrien Grand added a comment - It might be related to the fact that ScoreJoinQParserPlugin was put in a different package in 5.x compared to trunk? (my IDE complains that the package declaration does not match the folder the class is in)
          Hide
          Uwe Schindler added a comment -

          This happens under the following condition:

          • The release TAR file has correct javadocs, all links will be correct for the occifial release (they user http://lucene.apache.org/core/5_3_0/join/...)
          • Smoke tester fails when it unpacks the SRC-TGZ file (solr-5.3.0-src.tgz) and then runs ant documentation on it. In this case, the cross project links are static filesystem links (as seen above). Unfortunately, under this condition it seems to choose the wrong package. Maybe its a problem of order? I have the feeling we may have the same package in core and join that leads to the fact that it uses wrong module? It may also caused by not fully built Lucene" javadocs?

          Did you try the following;

          • ant clean
          • cd solr
          • ant javadocs

          (to isolate the whole thing out of full build). To me it looks like smoketester does not generate the full Lucene Javadocs, so it fails (see https://builds.apache.org/job/Lucene-Solr-SmokeRelease-5.x/ws/lucene/build/smokeTestRelease/tmp/unpack/solr-5.3.0/javadocs.log). So this could be a dependency problem. When solr builds javadocs it missies to trigger building of Lucenes Join.

          Unfortunately I cannot take care (vacation from today) and the release of 5.3.0 is close, so I would suggest to remove the cross module links for now and open separate issue to fix this.

          Show
          Uwe Schindler added a comment - This happens under the following condition: The release TAR file has correct javadocs, all links will be correct for the occifial release (they user http://lucene.apache.org/core/5_3_0/join/ ...) Smoke tester fails when it unpacks the SRC-TGZ file (solr-5.3.0-src.tgz) and then runs ant documentation on it. In this case, the cross project links are static filesystem links (as seen above). Unfortunately, under this condition it seems to choose the wrong package. Maybe its a problem of order? I have the feeling we may have the same package in core and join that leads to the fact that it uses wrong module? It may also caused by not fully built Lucene" javadocs? Did you try the following; ant clean cd solr ant javadocs (to isolate the whole thing out of full build). To me it looks like smoketester does not generate the full Lucene Javadocs, so it fails (see https://builds.apache.org/job/Lucene-Solr-SmokeRelease-5.x/ws/lucene/build/smokeTestRelease/tmp/unpack/solr-5.3.0/javadocs.log ). So this could be a dependency problem. When solr builds javadocs it missies to trigger building of Lucenes Join. Unfortunately I cannot take care (vacation from today) and the release of 5.3.0 is close, so I would suggest to remove the cross module links for now and open separate issue to fix this.
          Hide
          Uwe Schindler added a comment -

          Found the problem: Solr's common-build misses to build Join's Javadocs:

            <!-- dependency to ensure all lucene javadocs are present -->
            <target name="lucene-javadocs" depends="javadocs-lucene-core,javadocs-analyzers-common,javadocs-analyzers-icu,javadocs-analyzers-kuromoji,javadocs-analyzers-phonetic,javadocs-analyzers-smartcn,javadocs-analyzers-morfologik,javadocs-analyzers-stempel,javadocs-analyzers-uima,javadocs-backward-codecs,javadocs-codecs,javadocs-expressions,javadocs-suggest,javadocs-grouping,javadocs-queries,javadocs-queryparser,javadocs-highlighter,javadocs-memory,javadocs-misc,javadocs-spatial,javadocs-test-framework"/>
          
          Show
          Uwe Schindler added a comment - Found the problem: Solr's common-build misses to build Join's Javadocs: <!-- dependency to ensure all lucene javadocs are present --> <target name= "lucene-javadocs" depends= "javadocs-lucene-core,javadocs-analyzers-common,javadocs-analyzers-icu,javadocs-analyzers-kuromoji,javadocs-analyzers-phonetic,javadocs-analyzers-smartcn,javadocs-analyzers-morfologik,javadocs-analyzers-stempel,javadocs-analyzers-uima,javadocs-backward-codecs,javadocs-codecs,javadocs-expressions,javadocs-suggest,javadocs-grouping,javadocs-queries,javadocs-queryparser,javadocs-highlighter,javadocs-memory,javadocs-misc,javadocs-spatial,javadocs-test-framework" />
          Hide
          Uwe Schindler added a comment -

          This patch fixes the problem. Will commit this now.

          Show
          Uwe Schindler added a comment - This patch fixes the problem. Will commit this now.
          Hide
          ASF subversion and git services added a comment -

          Commit 1693197 from Uwe Schindler in branch 'dev/trunk'
          [ https://svn.apache.org/r1693197 ]

          SOLR-6234: Fix Javadocs dependency to join module

          Show
          ASF subversion and git services added a comment - Commit 1693197 from Uwe Schindler in branch 'dev/trunk' [ https://svn.apache.org/r1693197 ] SOLR-6234 : Fix Javadocs dependency to join module
          Hide
          ASF subversion and git services added a comment -

          Commit 1693198 from Uwe Schindler in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1693198 ]

          Merged revision(s) 1693197 from lucene/dev/trunk:
          SOLR-6234: Fix Javadocs dependency to join module

          Show
          ASF subversion and git services added a comment - Commit 1693198 from Uwe Schindler in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1693198 ] Merged revision(s) 1693197 from lucene/dev/trunk: SOLR-6234 : Fix Javadocs dependency to join module
          Hide
          Uwe Schindler added a comment -

          Fixed (hopefully) - waiting for next nightly run. If it still does not work, I hopy you can fix it based on my patch.

          Show
          Uwe Schindler added a comment - Fixed (hopefully) - waiting for next nightly run. If it still does not work, I hopy you can fix it based on my patch.
          Hide
          Adrien Grand added a comment -

          Thanks Uwe!

          Show
          Adrien Grand added a comment - Thanks Uwe!
          Hide
          Uwe Schindler added a comment - - edited

          Seems to pass now: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-trunk/245/console

          The problem was just that smoke tester does the good test to build the Solr javadocs in "isolation". The default builds don't trigger this, because they only check the javadocs as a "whole".

          Show
          Uwe Schindler added a comment - - edited Seems to pass now: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-trunk/245/console The problem was just that smoke tester does the good test to build the Solr javadocs in "isolation". The default builds don't trigger this, because they only check the javadocs as a "whole".
          Hide
          Uwe Schindler added a comment -

          ...out for vacation

          Show
          Uwe Schindler added a comment - ...out for vacation
          Hide
          Mikhail Khludnev added a comment -

          Uwe Schindler, danke schön! Sorry, guys. I knew I broke something by my first commit.

          Show
          Mikhail Khludnev added a comment - Uwe Schindler , danke schön! Sorry, guys. I knew I broke something by my first commit.
          Hide
          Adrien Grand added a comment -

          Mikhail, was it intentional to move ScoreJoinQParserPlugin to a different package when backporting? It's in oas.search in 5x and oas.search.join in trunk.

          Show
          Adrien Grand added a comment - Mikhail, was it intentional to move ScoreJoinQParserPlugin to a different package when backporting? It's in oas.search in 5x and oas.search.join in trunk.
          Hide
          Mikhail Khludnev added a comment -

          oh no. to clarify: I committed it to trunk first, then shoot my leg by old svn while merge into branch_5x.
          I prefer to move ScoreJoinQParserPlugin into oas.search.join at branch_5x if there is no objection. But would you tell me what I need to run before commit beside of test and precommit? Is prerelease worth to run?

          Show
          Mikhail Khludnev added a comment - oh no. to clarify: I committed it to trunk first, then shoot my leg by old svn while merge into branch_5x. I prefer to move ScoreJoinQParserPlugin into oas.search.join at branch_5x if there is no objection. But would you tell me what I need to run before commit beside of test and precommit? Is prerelease worth to run?
          Hide
          Mikhail Khludnev added a comment -

          precommit and nightly-smoke passed. Committing

          D       solr/core/src/java/org/apache/solr/search/ScoreJoinQParserPlugin.java
                  > moved to solr/core/src/java/org/apache/solr/search/join/ScoreJoinQParserPlugin.java
          A  +    solr/core/src/java/org/apache/solr/search/join/ScoreJoinQParserPlugin.java
                  > moved from solr/core/src/java/org/apache/solr/search/ScoreJoinQParserPlugin.java
          

          into ^/lucene/dev/branches/branch_5x

          Show
          Mikhail Khludnev added a comment - precommit and nightly-smoke passed. Committing D solr/core/src/java/org/apache/solr/search/ScoreJoinQParserPlugin.java > moved to solr/core/src/java/org/apache/solr/search/join/ScoreJoinQParserPlugin.java A + solr/core/src/java/org/apache/solr/search/join/ScoreJoinQParserPlugin.java > moved from solr/core/src/java/org/apache/solr/search/ScoreJoinQParserPlugin.java into ^/lucene/dev/branches/branch_5x
          Hide
          Uwe Schindler added a comment -

          No worries! This would have happened with me, too. This is just horrible dependency stuff and crazy javadocs & linter.

          See it like that: It wook me half an hour to find out what's going on. After reading Smoke Tester source and seeing that it does "ant clean" and then "ant javadocs" in Solr only, it was clear to me what's wrong - Solr was not building all required Lucene Javadocs before creating their own!

          Show
          Uwe Schindler added a comment - No worries! This would have happened with me, too. This is just horrible dependency stuff and crazy javadocs & linter. See it like that: It wook me half an hour to find out what's going on. After reading Smoke Tester source and seeing that it does "ant clean" and then "ant javadocs" in Solr only, it was clear to me what's wrong - Solr was not building all required Lucene Javadocs before creating their own!
          Hide
          Uwe Schindler added a comment -

          run before commit beside of test and precommit? Is prerelease worth to run?

          Precommit is fine. nightly-smoke is only needed for real structural changes. In fact, the Javadocs changes were structural So whenever you touch the build.xmls it may be good to run nightly-smoke, but in most cases I just wait for Jenkins to fail

          Show
          Uwe Schindler added a comment - run before commit beside of test and precommit? Is prerelease worth to run? Precommit is fine. nightly-smoke is only needed for real structural changes. In fact, the Javadocs changes were structural So whenever you touch the build.xmls it may be good to run nightly-smoke, but in most cases I just wait for Jenkins to fail
          Hide
          ASF subversion and git services added a comment -

          Commit 1693343 from mkhl@apache.org in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1693343 ]

          SOLR-6234: move ScoreJoinQParserPlugin.java into oas.search.join

          Show
          ASF subversion and git services added a comment - Commit 1693343 from mkhl@apache.org in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1693343 ] SOLR-6234 : move ScoreJoinQParserPlugin.java into oas.search.join
          Show
          Mikhail Khludnev added a comment - Adrien Grand Uwe Schindler thanks for your help! Timothy Potter , Cassandra Targett , I hit cwiki, would you mind to check: https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=32604257&selectedPageVersions=50&selectedPageVersions=49 https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=32604277&selectedPageVersions=71&selectedPageVersions=70
          Hide
          Cassandra Targett added a comment -

          Mikhail Khludnev. First, thanks so much for proactively updating the Ref Guide with your changes.

          I hope you don't mind too much, but I modified your text in a few ways:

          • The online version of the Ref Guide is always for the upcoming release of Solr, and as such we don't spend a lot of time describing how features used to work. While I understand that causes some confusion from time-to-time, a major problem with the old Solr Wiki was the attempt to maintain documentation for older versions of Solr along with the changes along the way. Personally, I find no harm mentioning when a feature was introduced, but at that point our convention is to remove documentation for the old behavior (unless, of course, the section is describing how to migrate to the new behavior).
          • With this general guideline, on the Uploading Data with Index Handlers page I flipped the emphasis to note the need to use the score parameter when deleting with the Join QP to avoid the error, and omitted the reference to 5.3 because the guide is for 5.3 already, and the section we're pointing to says the param was introduced in 5.3.
          • I re-worded the text a bit to provide more context for a less-expert user.
          • Standardized phrasings - i.e., "plain Solr" is usually referred to as "standalone Solr".
          Show
          Cassandra Targett added a comment - Mikhail Khludnev . First, thanks so much for proactively updating the Ref Guide with your changes. I hope you don't mind too much, but I modified your text in a few ways: The online version of the Ref Guide is always for the upcoming release of Solr, and as such we don't spend a lot of time describing how features used to work. While I understand that causes some confusion from time-to-time, a major problem with the old Solr Wiki was the attempt to maintain documentation for older versions of Solr along with the changes along the way. Personally, I find no harm mentioning when a feature was introduced, but at that point our convention is to remove documentation for the old behavior (unless, of course, the section is describing how to migrate to the new behavior). With this general guideline, on the Uploading Data with Index Handlers page I flipped the emphasis to note the need to use the score parameter when deleting with the Join QP to avoid the error, and omitted the reference to 5.3 because the guide is for 5.3 already, and the section we're pointing to says the param was introduced in 5.3. I re-worded the text a bit to provide more context for a less-expert user. Standardized phrasings - i.e., "plain Solr" is usually referred to as "standalone Solr".
          Hide
          Mikhail Khludnev added a comment -

          Chris Hostetter (Unused) does it make sense?

          Show
          Mikhail Khludnev added a comment - Chris Hostetter (Unused) does it make sense?
          Hide
          Mikhail Khludnev added a comment -

          Impl notes:

          • toString() is a little bit odd, it looks like
              "debug": {
                "rawquerystring": "{!join from=id to=manu_id_s score=max fromIndex=products}inc",
                "querystring": "{!join from=id to=manu_id_s score=max fromIndex=products}inc",
                "parsedquery": "OtherCoreJoinQuery(OtherCoreJoinQuery [fromIndex=products, fromCoreOpenTime=1438087133575 extends SameCoreJoinQuery [fromQuery=text:inc, fromField=id, toField=manu_id_s, scoreMode=Max]])",
                "parsedquery_toString": "OtherCoreJoinQuery [fromIndex=products, fromCoreOpenTime=1438087133575 extends SameCoreJoinQuery [fromQuery=text:inc, fromField=id, toField=manu_id_s, scoreMode=Max]]",
                "explain": {
            

            Let me know if you need to change it.

          • for cross core join: when you commit into fromIndex query cache entry equality is determined by coreOpenTime() which is obtained from System.currentTimeMillis() it might cause some issues on edge cases/plarforms (that's why it wasn't covered with test), but this approach was inherited from {!join}. Raise an issue if it's significant to you.
          Show
          Mikhail Khludnev added a comment - Impl notes: toString() is a little bit odd, it looks like "debug" : { "rawquerystring" : "{!join from=id to=manu_id_s score=max fromIndex=products}inc" , "querystring" : "{!join from=id to=manu_id_s score=max fromIndex=products}inc" , "parsedquery" : "OtherCoreJoinQuery(OtherCoreJoinQuery [fromIndex=products, fromCoreOpenTime=1438087133575 extends SameCoreJoinQuery [fromQuery=text:inc, fromField=id, toField=manu_id_s, scoreMode=Max]])" , "parsedquery_toString" : "OtherCoreJoinQuery [fromIndex=products, fromCoreOpenTime=1438087133575 extends SameCoreJoinQuery [fromQuery=text:inc, fromField=id, toField=manu_id_s, scoreMode=Max]]" , "explain" : { Let me know if you need to change it. for cross core join: when you commit into fromIndex query cache entry equality is determined by coreOpenTime() which is obtained from System.currentTimeMillis() it might cause some issues on edge cases/plarforms (that's why it wasn't covered with test), but this approach was inherited from {!join}. Raise an issue if it's significant to you.
          Hide
          ASF subversion and git services added a comment -

          Commit 1693760 from mkhl@apache.org in branch 'dev/trunk'
          [ https://svn.apache.org/r1693760 ]

          SOLR-6234: moving lines in CHANGES.txt

          Show
          ASF subversion and git services added a comment - Commit 1693760 from mkhl@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1693760 ] SOLR-6234 : moving lines in CHANGES.txt
          Hide
          Shalin Shekhar Mangar added a comment -

          Bulk close for 5.3.0 release

          Show
          Shalin Shekhar Mangar added a comment - Bulk close for 5.3.0 release

            People

            • Assignee:
              Mikhail Khludnev
              Reporter:
              Mikhail Khludnev
            • Votes:
              4 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development