Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-2146

Storm SQL - Support JOIN between Streaming data source and State data source in Trident mode

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Invalid
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: storm-sql
    • Labels:
      None

      Description

      This issue is only targeted to Trident mode.

      This issue also handles drop supporting join between Streaming data source and Streaming data source since the join is based on micro-batch so result is non-deterministic which doesn't make sense.

      This issue tracks supporting join between Streaming data source and State data source.

      Following works should be done:

      • create interface for State Table which is able to query value based on key
      • define DDL for State data source
      • modify visitTableScan in TridentLogicalPlanCompiler to handle State data source
      • modify visitJoin in TridentLogicalPlanCompiler
        • check join condition: first one is Streaming and next one is State
          • table order can be reversible
          • only simple equi-join is supported (support compose key?)
        • query to State via join key and join Streaming row and the value
      • address unit tests
      • update document

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              kabhwan Jungtaek Lim
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: