Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-11026

Rework creation of sql-client connector/format jars

    XMLWordPrintableJSON

    Details

    • Release Note:
      Hide
      The naming scheme for kafka/elasticsearch6 sql-jars has been changed.

      In maven terms, they no longer have the "sql-jar" qualifier and the artifactId is now prefixed with "flink-sql" instead of "flink", e.g., "flink-sql-connector-kafka...".
      Show
      The naming scheme for kafka/elasticsearch6 sql-jars has been changed. In maven terms, they no longer have the "sql-jar" qualifier and the artifactId is now prefixed with "flink-sql" instead of "flink", e.g., "flink-sql-connector-kafka...".

      Description

      For the SQL client we currently have a separate sql-jars profile in various connectors/formats that create an additional fat jar with a separate classifier.

      One of the core maven mantras is "One artifact per module.", and we see the importance of this mantra as our current packaging strategy makes it impossible to provide different NOTICE files for the created jars (regular and sql-jar).

      Currently we would be forced to provide the same file for both jars, which will cause problems for any downstream users that wants to handle NOTICE files properly. We would cause the same issue we had with netty, which categorically claims to be bundling dependencies although it doesn't, forcing us to manually cut out the valid parts.

      My proposal is to move custom packaging strategies into their own module that depend on the original module.

       I will use flink-connector-elasticsearch6 as an example, which packages both a regular jar without any included dependencies, and a sql jar bundling everything.

      • create a separate flink-sql-connector-elasticsearch6/flink-connector-elasticsearch6-uber{{}} module
      • this module depends on flink-connector-elasticsearch6, and bundles all dependencies
      • move the current shading logic for the sql jar out of the sql-jars profile{{}}
      • add a sql-jars profile to flink-connectors for skipping the creation of these jars

        Attachments

          Activity

            People

            • Assignee:
              chesnay Chesnay Schepler
              Reporter:
              chesnay Chesnay Schepler
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 0.5h
                0.5h