Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-11095

Table based ML Pipeline

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • None
    • None

    Description

      As Table API will be the unified high level API for both batch and streaming, it is desired to have a table API based ML Pipeline definition. 

      This new table based ML Pipeline shall:

      1. support unified batch/stream ML/AI functionalities (train/inference).  
      2. seamless integrated with flink Table based ecosystem. 
      3. provide a base for further flink based AI platform/tooling support.
      4. provide a base for further flink ML SQL integration. 

      The initial design is here https://docs.google.com/document/d/1PLddLEMP_wn4xHwi6069f3vZL7LzkaP0MN9nAB63X90/edit?usp=sharing.

      And based on this design, I made some initial implementation/prototyping. I will share the code later.

      This is the umbrella JIRA. I will create corresponding sub-jira for each sub-task. 

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            whjiang Weihua Jiang
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment