[FLINK-2837] FlinkTopologyBuilder cannot handle multiple input streams

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Component/s: Legacy Components / Storm Compatibility
• Assignee: Matthias J. Sax
• Reporter: Matthias J. Sax
• Created: 08/Oct/15 14:09
• Updated: 28/Dec/15 14:15
• Resolved: 28/Dec/15 10:56

Description

FlinkTopologyBuilder cannot handle multiple input streams correctly. Instead of union the incoming streams, it replicates the consuming bolt and each (logical) instance processes one of the input streams.

For example:

final FlinkTopologyBuilder builder = new FlinkTopologyBuilder();

builder.setSpout(spoutId1, new FiniteRandomSpout(0, 10));
builder.setSpout(spoutId2, new FiniteRandomSpout(1, 8));
builder.setSpout(spoutId3, new FiniteRandomSpout(2, 13));

builder.setBolt(boltId, new MergerBolt())
	.shuffleGrouping(spoutId1)
	.shuffleGrouping(spoutId2)
	.shuffleGrouping(spoutId3);

builder.setBolt("sink", new BoltPrintSink(new SimpleOutputFormatter()))
	.shuffleGrouping(boltId);

will only print the data from a single source instead of all sources.