[STORM-2824] Ability to configure topologies for exactly once processing

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.0.1
• Component/s: storm-core
• Environment: CentOS 7, Docker
• Assignee: Unassigned
• Reporter: Anton Alfred
• Created: 19/Nov/17 04:48
• Updated: 20/Nov/17 03:47

Description

The default implementation of a spout (Kafka) is to wait for acknowledgement, if an acknowledgement is not provided the tuple is replayed leading to an at least once processing model.

Can an option be provided to always acknowledge even in the event of error in any spout or bolt and the user decide which mode the topology should be configured.

There are cases like multiple bolts (B) inserting to persistent stores (PS) like B1 - PS1, B2-PS2, B3-PS3, the fact that B2-PS2 bolt fail doesn't mean that the tuple needs to be replayed leading to complexity on the logic of bolts, it would be easier if this was configurable and the user of the topology decides which style to choose.