Create a general purpose windowed bolt that performs Joins on multiple data streams.
Since, depending on the topo config, the bolt could be receiving data either on 'default' streams or on named streams .... join bolt should be able to differentiate the incoming data based on names of upstream components as well as stream names.
The following SQL style join involving 4 tables :
Could be expressed using the Join Bolt over 4 named streams as :
Or based on named source components :
In order for the tuples to be joined correctly, 'fields grouping' should be employed on the incoming streams. Each stream should be grouped on the same key using which it will be joined against other streams. This is a restriction compared to SQL which allows join a table with others on any key and any number of keys.
For example: If a 'Stream1' is Fields Grouped on 'key1', we cannot use a different 'key2' on 'Stream1' to join it with other streams. However, 'Stream1' can be joined using the same key with multiple other streams as show in this SQL.
Consequently the join bolt's syntax is a bit simplified compared to SQL. The key name for any given stream only appears once, as soon the stream is introduced for the first time in the join. Thereafter that key is implicitly used for joining. See the case of 'stream3' being joined with both 'stream2' and 'stream4' in the first example.