Details
-
Improvement
-
Status: To Do
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
Currently in the parser aggregation, the aggregate bolt doesn't do any clever routing for chained parsers. It just routes messages to a Kafka topic for the sensors, even when they're already bundled into the aggregated bolt.
An option to skip the intermediate Kafka topic should be added. This should preferably be optional on a per sensor basic, e.g. If parser A produces documents B,C, and D, we may choose to have B and C skip intermediate Kafka and D use Kafka. This is useful in cases where, for whatever reason, we choose to have parser D be in it's own topology (i.e. parsers A,B,C are in an aggregated topology, and D is in a single topology. Even though parser A produces all document B,C, and D).