Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-11063

Make flink-table Scala-free

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.7.0
    • Fix Version/s: None
    • Component/s: Table SQL / API
    • Labels:
      None

      Description

      Currently, the Table & SQL API is implemented in Scala. This decision was made a long-time ago when the initial code base was created as part of a master's thesis. The community kept Scala because of the nice language features that enable a fluent Table API like table.select('field.trim()) and because Scala allows for quick prototyping (e.g. multi-line comments for code generation). The committers enforced not splitting the code-base into two programming languages.

      However, nowadays the flink-table module more and more becomes an important part in the Flink ecosystem. Connectors, formats, and SQL client are actually implemented in Java but need to interoperate with flink-table which makes these modules dependent on Scala. As mentioned in an earlier mail thread, using Scala for API classes also exposes member variables and methods in Java that should not be exposed to users. Java is still the most important API language and right now we treat it as a second-class citizen.

      In order to not introduce more technical debt, the community aims to make the flink-table module Scala-free as a long-term goal. This will be a continuous effort that can not be finished within one release. We aim for avoiding API-breaking changes.

      A full description can be found in the corresponding FLIP-28.

      FLIP-28 also contains a rough roadmap and serves as migration guidelines.

      This Jira issue is an umbrella issue for tracking the efforts and possible migration blockers.

      Update: Due to the big code contribution of Alibaba into Flink SQL. We will only perform porting of API classes for now. This is mostly tracked by FLINK-11448.

      FLIP-28 is legacy and has been integrated into FLIP-32.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                twalthr Timo Walther
              • Votes:
                0 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated: