[SOLR-9559] Add ExecutorStream to execute stored Streaming Expressions

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Fix Version/s: 6.3
• Assignee: Joel Bernstein
• Reporter: Joel Bernstein
• Created: 25/Sep/16 17:38
• Updated: 08/Oct/19 14:50
• Resolved: 27/Oct/16 03:16

Description

The ExecutorStream will wrap a stream which contains Tuples with Streaming Expressions to execute. By default the ExecutorStream will look for the expression in the expr_s field in the Tuples.

The ExecutorStream will have an internal thread pool so expressions can be executed in parallel on a single worker. The ExecutorStream can also be wrapped by the parallel function to partition the Streaming Expressions that need to be executed across a cluster of worker nodes.

Sample syntax:

daemon(executor(threads=10, topic(storedExpressions, fl="expr_s", ...)))

In the example above a daemon wraps an executor which wraps a topic that is reading stored Streaming Expressions. The daemon will call the executor at intervals which will execute all the expressions retrieved by the topic.

Issue Links

• depends upon: SOLR-9387 Allow topic expression to store queries and macros
• is related to: SOLR-9684 Add priority Streaming Expression
• supercedes: SOLR-9387 Allow topic expression to store queries and macros