[STORM-2423] Join Bolt : Use explicit instead of default window anchoring of emitted tuples

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.0.0, 1.x
• Fix Version/s: 2.0.0, 1.1.0
• Component/s: storm-core
• Assignee: Roshan Naik
• Reporter: Roshan Naik
• Created: 18/Mar/17 02:44
• Updated: 21/Mar/17 05:24
• Resolved: 21/Mar/17 05:15

Description

Default anchoring will anchor each emitted tuple to every tuple in current window. This requires a very large numbers of ACKs from any downstream bolt. If topology.debug is enabled, it also worsens the load on the system significantly.

Letting the topo run in this mode (in particular with max.spout.pending disabled), could lead to the worker running out of memory and crashing.

Fix: Join Bolt should avoid using default window anchoring, and explicitly anchor each emitted tuple with the exact matching tuples form each inputs streams. This reduces the complexity of the tuple trees and consequently the reduces burden on the ACKing & messaging subsystems.

Issue Links

• links to: GitHub Pull Request #2019