[STORM-2146] Storm SQL - Support JOIN between Streaming data source and State data source in Trident mode

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Invalid
• Component/s: storm-sql
• Epic Link: Storm SQL - expand supporting external components
• Assignee: Unassigned
• Reporter: Jungtaek Lim
• Created: 11/Oct/16 03:10
• Updated: 17/Nov/16 15:08
• Resolved: 17/Nov/16 15:08

Description

This issue is only targeted to Trident mode.

This issue also handles drop supporting join between Streaming data source and Streaming data source since the join is based on micro-batch so result is non-deterministic which doesn't make sense.

This issue tracks supporting join between Streaming data source and State data source.

Following works should be done:

• create interface for State Table which is able to query value based on key
• define DDL for State data source
• modify visitTableScan in TridentLogicalPlanCompiler to handle State data source
• modify visitJoin in TridentLogicalPlanCompiler
  • check join condition: first one is Streaming and next one is State
    • table order can be reversible
    • only simple equi-join is supported (support compose key?)
  • query to State via join key and join Streaming row and the value
• address unit tests
• update document