[SOLR-8188] Add hash style joins to the Streaming API and Streaming Expressions

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Implemented
• Fix Version/s: 6.0
• Component/s: SolrJ
• Assignee: Dennis Gove
• Reporter: Dennis Gove
• Created: 23/Oct/15 01:20
• Updated: 09/May/16 18:46
• Resolved: 12/Nov/15 00:22

Description

Add HashJoinStream and OuterHashJoinStream to the Streaming API to allow for optimized joining between sub-streams.

HashJoinStream is similar to an InnerJoinStream except that it does not insist on any particular order and will read all values from the stream being hashed (hashStream) when open() is called. During read() it will return the next tuple from the stream not being hashed (fullStream) which has at least one matching record in hashStream. It will return a tuple which is the merge of both tuples. If the tuple from the fullStream matches with more than one tuple from the hashStream then calling read() will return the merge with the next matching tuple. The order of the resulting stream is the order of the fullStream.

OuterHashJoinStream is similar to a HashJoinStream and LeftOuterJoinStream in that a tuple from fullStream will be returned even if it doesn't have a matching record in hashStream. All other pieces are identical.

In expression form

hashJoin(
  search(collection1, q=*:*, fl="fieldA, fieldB, fieldC", ...),
  hashed=search(collection2, q=*:*, fl="fieldA, fieldB, fieldE", ...),
  on="fieldA, fieldB"
)

outerHashJoin(
  search(collection1, q=*:*, fl="fieldA, fieldB, fieldC", ...),
  hashed=search(collection2, q=*:*, fl="fieldA, fieldB, fieldE", ...),
  on="fieldA, fieldB"
)

As you can see the hashStream is named parameter which makes it very clear which stream should be hashed.

Issue Links

• is related to: SOLR-8125 Umbrella ticket for Streaming and SQL issues