[SOLR-8281] Add RollupMergeStream to Streaming API

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Assignee: Joel Bernstein
• Reporter: Joel Bernstein
• Created: 12/Nov/15 00:40
• Updated: 27/Jan/22 17:54
• Resolved: 27/Jan/22 17:54

Description

The RollupMergeStream merges the aggregate results emitted by the RollupStream on worker nodes.

This is designed to be used in conjunction with the HashJoinStream to perform rollup Aggregations on the joined Tuples. The HashJoinStream will require the tuples to be partitioned on the Join keys. To avoid needing to repartition on the group by fields for the RollupStream, we can perform a merge of the rolled up Tuples coming from the workers.

The construct would like this:

mergeRollup (...
                      parallel (...
                                    rollup (...
                                                hashJoin (
                                                                  search(...),
                                                                  search(...),
                                                                  on="fieldA" 
                                                )
                                     )
                         )
               )

The pseudo code above would push the hashJoin and rollup to the worker nodes. The emitted rolled up tuples would be merged by the mergeRollup.

Issue Links

• duplicates: SOLR-15036 Use plist automatically for executing a facet expression against a collection alias backed by multiple collections