Details
-
New Feature
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
A first minimum version of FLIP-24 for the upcoming
Flink SQL Client has been merged to the master. We also merged
possibilities to discover and configure table sources without a single
line of code using string-based properties and Java service provider
discovery.
We are now facing the issue of how to manage dependencies in this new
environment. It is different from how regular Flink projects are created
(by setting up a a new Maven project and build a jar or fat jar).
Ideally, a user should be able to select from a set of prepared
connectors, catalogs, and formats. E.g., if a Kafka connector and Avro
format is needed, all that should be required is to move a
"flink-kafka.jar" and "flink-avro.jar" into the "sql_lib" directory that
is shipped to a Flink cluster together with the SQL query.
As discussed on ML, we will build fat jars for these modules with every Flink release that can be hostet somewhere (e.g. Apache infrastructure, but not Maven central). This would make it very easy to add a dependency by downloading the prepared JAR files.
Attachments
1.
|
Create a SQL Client Kafka fat-jar |
|
Resolved | Timo Walther |
2.
|
Create a SQL Client JSON format fat-jar |
|
Resolved | Timo Walther |
3.
|
Create a SQL Client Avro format fat-jar |
|
Resolved | Timo Walther |