[SOLR-8208] DocTransformer executes sub-queries

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 6.1, 7.0
• Component/s: Response Writers
• Labels: features newbie
• Assignee: Mikhail Khludnev
• Reporter: Mikhail Khludnev
• Created: 25/Oct/15 21:56
• Updated: 03/Apr/20 19:57
• Resolved: 17/May/16 21:18

Description

The initial idea was to return "from" side of query time join via doctransformer. I suppose it isn't query-time join specific, thus let to specify any query and parameters for them, let's call it sub-query. But it might be problematic to escape subquery parameters, including local ones, e.g. what if subquery needs to specify own doctransformer in &fl=[..] ?
I suppose we can specify subquery parameter prefix:

..&q=name_s:john&fl=*,depts:[subquery fromIndex=departments]&
depts.q={!term f=dept_id_s v=$row.dept_ss_dv}&depts.fl=text_t,dept_id_s_dv&depts.rows=12&depts.sort=id desc

response is like

       
<response>
...
    <result name="response" numFound="1" start="0">
        <doc>
            <str name="id">1</str>
            <str name="name_s_dv">john</str>
..
            <result name="depts" numFound="2" start="0">
                <doc>
                    <str name="dept_id_s_dv">Engineering</str>
                    <str name="text_t">These guys develop stuff</str>
                </doc>
                <doc>
                    <str name="dept_id_s_dv">Support</str>
                    <str name="text_t">These guys help users</str>
                </doc>
            </result>
        </doc>
    </result>
</response>

• fl=depts:[subquery] executes a separate request for every query result row, and adds it into a document as a separate result list. The given field name (here it's 'depts') is used as a prefix to shift subquery parameters from main query parameter, eg depts.q turns to q for subquery, depts.rows to rows.
• document fields are available as implicit parameters with prefix row. eg. if result document has a field dept_id it can be referred as v=$row.dept_id this combines well with {!terms} query parser
• separator=',' is used when multiple field values are combined in parameter. eg. a document has multivalue field

  dept_ids={2,3}

  , thus referring to it via

  ..&dept.q={!terms f=id v=$row.dept_ids}&..

  executes a subquery

  {!terms f=id}2,3

  . When omitted it's a comma.

• fromIndex=othercore optional param allows to run subquery on other core, like it works on query time join
  However, it doesn't work on cloud setup (and will let you know), but it's proposed to use regular params (collection, shards - whatever, with subquery prefix as below ) to issue subquery to a collection

  q=name_s:dave&indent=true&fl=*,depts:[subquery]&rows=20&
  depts.q={!terms f=dept_id_s v=$row.dept_ss_dv}&depts.fl=text_t&
  depts.indent=true&
  depts.collection=departments&
  depts.rows=10&depts.logParamsList=q,fl,rows,row.dept_ss_dv
  

Caveat: it should be a way slow; it handles only search result page, not entire result set.

Issue Links

• causes: SOLR-14384 Stack SolrRequestInfo
• is broken by: SOLR-9448 [subquery] calls another collection fails with "undefined field" or NPE from mergeIds
• is related to: SOLR-10669 bulk [subquery] result transformer
• is related to: SOLR-10589 [subquery] debugQuery=true support
• is related to: SOLR-10673 [subquery] sort by geodist
• relates to: SOLR-10571 [subquery] requires field to be specified in both fl and subquery.fl to be responded in subquery results
• relates to: SOLR-10825 [subquery] facets support
• relates to: SOLR-9101 introduce SolrRequestInfo.doInSuspension(Callable<T>). make SolrRequestInfo final
• links to: inspired by [child]
• links to: spin off from the thread
• links to: wiki update

Sub-Tasks

• fromIndex= param for [subquery ] DocTransformer: Closed