[FLINK-11063] Make flink-table Scala-free

• Type: New Feature
• Status: Closed
• Priority: Not a Priority
• Resolution: Implemented
• Affects Version/s: 1.7.0
• Component/s: Table SQL / API
• Labels: auto-deprioritized-major auto-deprioritized-minor
• Assignee: Unassigned
• Reporter: Timo Walther
• Created: 04/Dec/18 10:30
• Updated: 25/Jan/22 12:11
• Resolved: 25/Jan/22 12:11

Description

Currently, the Table & SQL API is implemented in Scala. This decision was made a long-time ago when the initial code base was created as part of a master's thesis. The community kept Scala because of the nice language features that enable a fluent Table API like table.select('field.trim()) and because Scala allows for quick prototyping (e.g. multi-line comments for code generation). The committers enforced not splitting the code-base into two programming languages.

However, nowadays the flink-table module more and more becomes an important part in the Flink ecosystem. Connectors, formats, and SQL client are actually implemented in Java but need to interoperate with flink-table which makes these modules dependent on Scala. As mentioned in an earlier mail thread, using Scala for API classes also exposes member variables and methods in Java that should not be exposed to users. Java is still the most important API language and right now we treat it as a second-class citizen.

In order to not introduce more technical debt, the community aims to make the flink-table module Scala-free as a long-term goal. This will be a continuous effort that can not be finished within one release. We aim for avoiding API-breaking changes.

A full description can be found in the corresponding FLIP-28.

FLIP-28 also contains a rough roadmap and serves as migration guidelines.

This Jira issue is an umbrella issue for tracking the efforts and possible migration blockers.

Update: Due to the big code contribution of Alibaba into Flink SQL. We will only perform porting of API classes for now. This is mostly tracked by FLINK-11448.

FLIP-28 is legacy and has been integrated into FLIP-32.

Issue Links

• is a parent of: FLINK-11065 Migrate flink-table runtime classes
• is a parent of: FLINK-11066 Migrate main Table API classes to flink-table-api-base
• is a parent of: FLINK-11117 Migrate connector classes to flink-table-spi
• is a parent of: FLINK-10686 Introduce a flink-table-common module