[STORM-1873] Reemit late tuples in windowed mode

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0, 1.1.0
• Component/s: storm-core
• Epic Link: Release Apache Storm 1.1.0
• Assignee: Balazs Kossovics
• Reporter: Balazs Kossovics
• Created: 31/May/16 12:30
• Updated: 09/Jun/16 08:54
• Resolved: 09/Jun/16 05:59

Description

Currently late tuples are just logged (and acknowledged in the coming 1.0.2), but in our use-case it would be desirable to emit them on a different stream than the default.

I implemented a first version, where every windowed bolt are going to have a '_late' stream by default, and component-specific parameter (Config.TOPOLOGY_BOLTS_EMIT_LATE_TUPLE) the definer of the bolt could turn on or off the emission of the late tuples on this stream.

One could turn on the emission of late tuples with a builder method like this:

MyWindowedBolt.java

new MyWindowedBolt()
        .withTimestampField("timestamp")
        .withLateTupleEmission(true)
        .withWindow(
                new BaseWindowedBolt.Duration(1, TimeUnit.MINUTES),
                new BaseWindowedBolt.Duration(1, TimeUnit.SECONDS)
        );

What do you think about it?

Issue Links

• links to: GitHub Pull Request #1453