[STORM-3646] Flush only happens on executor main thread

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Abandoned
• Assignee: Unassigned
• Reporter: Ethan Li
• Created: 04/Jun/20 16:50
• Updated: 02/Jul/24 09:54
• Resolved: 02/Jul/24 09:54

Description

Flush only happens on executor main thread.
https://github.com/apache/storm/blob/v2.1.0/storm-client/src/jvm/org/apache/storm/executor/ExecutorTransfer.java#L70-L74

If topologies have their own threads within bolts, and when

topology.producer.batch.size
topology.transfer.batch.size

are larger than 1, some tuples will be batched and possible never be sent to downstream.

The default value for them is 1. So it is fine when the configs are not explicitly changed.

Issue Links

• relates to: STORM-3620 Data corruption can happen when components are multi-threaded because of non thread-safe serializer