Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.8.0
    • Fix Version/s: 1.8.0
    • Component/s: Release System
    • Labels:
      None
    • Release Note:
      Hide
      Convenience binaries that include hadoop are no longer released.

      If a deployment relies on flink-shaded-hadoop2 being included in flink-dist, then it must be manually downloaded and copied into the /lib directory.
      Alternatively, a Flink distribution that includes hadoop can be built by packaging flink-dist and activating the "include-hadoop" maven profile.

      As hadoop is no longer included in flink-dist by default, specifying "-DwithoutHadoop" when packaging flink-dist no longer impacts the build.
      Show
      Convenience binaries that include hadoop are no longer released. If a deployment relies on flink-shaded-hadoop2 being included in flink-dist, then it must be manually downloaded and copied into the /lib directory. Alternatively, a Flink distribution that includes hadoop can be built by packaging flink-dist and activating the "include-hadoop" maven profile. As hadoop is no longer included in flink-dist by default, specifying "-DwithoutHadoop" when packaging flink-dist no longer impacts the build.

      Description

      Currently we release 10 different binary releases (2 scala versions * (4 hadoop version + hadoop-free)), which has increased the size of our release to more than 2 GB.
      Naturally, building Flink 10 times also takes a while, slowing down the release process.

      However, the only difference between the hadoop versions is the bundled flink-shaded-hadoop2 jar; the rest is completely identical.

      I propose to stop releasing hadoop-specific distributions, and instead have us release multiple versions of flink-shaded-hadoop2 that users copy into the hadoop-free distribution if required.

        Attachments

          Activity

            People

            • Assignee:
              Zentol Chesnay Schepler
              Reporter:
              Zentol Chesnay Schepler
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 1h 20m
                1h 20m