Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-19976

FLIP-136: Improve interoperability between DataStream and Table API

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Table SQL / API
    • Labels:
      None

      Description

      The Table API has received many new features over the last year. It supports a new type system (FLIP-37), connectors support changelogs (FLIP-95), we have well defined internal data structures (FLIP-95), support for result retrieval in an interactive fashion (FLIP-84), and soon new TableDescriptors (FLIP-129).

      However, the interfaces from and to DataStream API have not been touched during the introduction of these new features and are kind of outdated. The interfaces lack important functionality that is available in Table API but not exposed to DataStream API users. DataStream API is still our most important API which is why a good interoperability is crucial.

      This FLIP is a mixture of different topics that improve the interoperability between DataStream and Table API in terms of:

      • DataStream ↔ Table conversion
      • translation of type systems TypeInformation ↔ DataType
      • schema definition (incl. rowtime, watermarks, primary key)
      • changelog handling
      • row handling in DataStream API

      See:
      https://cwiki.apache.org/confluence/pages/resumedraft.action?draftId=158872537&draftShareId=69731f11-580f-4bfe-847f-e5eb3e585f1f&

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                twalthr Timo Walther
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated: