Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-8188

Add hash style joins to the Streaming API and Streaming Expressions

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Implemented
    • Affects Version/s: None
    • Fix Version/s: 6.0
    • Component/s: SolrJ
    • Labels:
      None

      Description

      Add HashJoinStream and OuterHashJoinStream to the Streaming API to allow for optimized joining between sub-streams.

      HashJoinStream is similar to an InnerJoinStream except that it does not insist on any particular order and will read all values from the stream being hashed (hashStream) when open() is called. During read() it will return the next tuple from the stream not being hashed (fullStream) which has at least one matching record in hashStream. It will return a tuple which is the merge of both tuples. If the tuple from the fullStream matches with more than one tuple from the hashStream then calling read() will return the merge with the next matching tuple. The order of the resulting stream is the order of the fullStream.

      OuterHashJoinStream is similar to a HashJoinStream and LeftOuterJoinStream in that a tuple from fullStream will be returned even if it doesn't have a matching record in hashStream. All other pieces are identical.

      In expression form

      hashJoin(
        search(collection1, q=*:*, fl="fieldA, fieldB, fieldC", ...),
        hashed=search(collection2, q=*:*, fl="fieldA, fieldB, fieldE", ...),
        on="fieldA, fieldB"
      )
      
      outerHashJoin(
        search(collection1, q=*:*, fl="fieldA, fieldB, fieldC", ...),
        hashed=search(collection2, q=*:*, fl="fieldA, fieldB, fieldE", ...),
        on="fieldA, fieldB"
      )
      

      As you can see the hashStream is named parameter which makes it very clear which stream should be hashed.

        Attachments

        1. SOLR-8188.patch
          25 kB
          Dennis Gove
        2. SOLR-8188.patch
          25 kB
          Dennis Gove
        3. SOLR-8188.patch
          25 kB
          Dennis Gove

          Issue Links

            Activity

              People

              • Assignee:
                dpgove Dennis Gove
                Reporter:
                dpgove Dennis Gove
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: