Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-23986

Add support for opting-out of Scala

    XMLWordPrintableJSON

Details

    • Technical Debt
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.15.0
    • None
    • Hide
      The Java DataSet/-Stream APIs are now independent from Scala and no longer transitively depend on it.

      The implications are the following:

      a)
      If you only intend to use the Java APIs, with Java types,
      then you can opt-in to a Scala-free Flink by removing the flink-scala jar from the lib/ directory of the distribution.
      You are then free to use any Scala version and Scala libraries.
      You can either bundle Scala itself in your user-jar; or put into the lib/ directory of the distribution.

      b)
      If you relied on the Scala APIs, without an explicit dependency on them, then you may experience issues when building your projects.
      You can solve this by adding explicit dependencies to the APIs that you are using. This should primarily affect users of the Scala DataStream/CEP APIs.

      c)
      A large number of modules have lost their Scala suffix.
      Any dependency on one of the modules listed below needs to be updated.
      Further caution is advised when mixing dependencies from different Flink versions (e.g., an older connector),
      as you may now end up pulling in multiple versions of a single module (that would previously be prevented by the name being equal).

      Any dependency to one of the following modules needs to be updated to no longer include a suffix:
      flink-cep
      flink-clients
      flink-connector-elasticsearch-base
      flink-connector-elasticsearch5
      flink-connector-elasticsearch6
      flink-connector-elasticsearch7
      flink-connector-gcp-pubsub
      flink-connector-hbase-1.4
      flink-connector-hbase-2.2
      flink-connector-hbase-base
      flink-connector-jdbc
      flink-connector-kafka
      flink-connector-kinesis
      flink-connector-nifi
      flink-connector-pulsar
      flink-connector-rabbitmq
      flink-connector-testing
      flink-connector-twitter
      flink-connector-wikiedits
      flink-container
      flink-dstl-dfs
      flink-gelly
      flink-hadoop-bulk
      flink-kubernetes
      flink-runtime-web
      flink-sql-connector-elasticsearch6
      flink-sql-connector-elasticsearch7
      flink-sql-connector-hbase-1.4
      flink-sql-connector-hbase-2.2
      flink-sql-connector-kafka
      flink-sql-connector-kinesis
      flink-sql-connector-rabbitmq
      flink-state-processor-api
      flink-statebackend-rocksdb
      flink-streaming-java
      flink-table-api-java-bridge
      flink-test-utils
      flink-yarn
      Show
      The Java DataSet/-Stream APIs are now independent from Scala and no longer transitively depend on it. The implications are the following: a) If you only intend to use the Java APIs, with Java types, then you can opt-in to a Scala-free Flink by removing the flink-scala jar from the lib/ directory of the distribution. You are then free to use any Scala version and Scala libraries. You can either bundle Scala itself in your user-jar; or put into the lib/ directory of the distribution. b) If you relied on the Scala APIs, without an explicit dependency on them, then you may experience issues when building your projects. You can solve this by adding explicit dependencies to the APIs that you are using. This should primarily affect users of the Scala DataStream/CEP APIs. c) A large number of modules have lost their Scala suffix. Any dependency on one of the modules listed below needs to be updated. Further caution is advised when mixing dependencies from different Flink versions (e.g., an older connector), as you may now end up pulling in multiple versions of a single module (that would previously be prevented by the name being equal). Any dependency to one of the following modules needs to be updated to no longer include a suffix: flink-cep flink-clients flink-connector-elasticsearch-base flink-connector-elasticsearch5 flink-connector-elasticsearch6 flink-connector-elasticsearch7 flink-connector-gcp-pubsub flink-connector-hbase-1.4 flink-connector-hbase-2.2 flink-connector-hbase-base flink-connector-jdbc flink-connector-kafka flink-connector-kinesis flink-connector-nifi flink-connector-pulsar flink-connector-rabbitmq flink-connector-testing flink-connector-twitter flink-connector-wikiedits flink-container flink-dstl-dfs flink-gelly flink-hadoop-bulk flink-kubernetes flink-runtime-web flink-sql-connector-elasticsearch6 flink-sql-connector-elasticsearch7 flink-sql-connector-hbase-1.4 flink-sql-connector-hbase-2.2 flink-sql-connector-kafka flink-sql-connector-kinesis flink-sql-connector-rabbitmq flink-state-processor-api flink-statebackend-rocksdb flink-streaming-java flink-table-api-java-bridge flink-test-utils flink-yarn

    Description

      The goal of this ticket is to fully isolate the Java APIs (excluding the Table API) from Scala, such that jobs that don't require Scala can be run without Scala being in lib/.
      In particular we want to allow Scala users that use the Java APIs to use whichever Scala version they want to use.

      Attachments

        Issue Links

          Activity

            People

              chesnay Chesnay Schepler
              chesnay Chesnay Schepler
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: