[FLINK-11266] Only release hadoop-free Flink

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.8.0
• Fix Version/s: 1.8.0
• Component/s: Release System

Release Note

Convenience binaries that include hadoop are no longer released.

If a deployment relies on flink-shaded-hadoop2 being included in flink-dist, then it must be manually downloaded and copied into the /lib directory.
Alternatively, a Flink distribution that includes hadoop can be built by packaging flink-dist and activating the "include-hadoop" maven profile.

As hadoop is no longer included in flink-dist by default, specifying "-DwithoutHadoop" when packaging flink-dist no longer impacts the build.

• Assignee: Chesnay Schepler
• Reporter: Chesnay Schepler
• Created: 04/Jan/19 11:00
• Updated: 29/Jan/19 15:01
• Resolved: 29/Jan/19 15:01

Description

Currently we release 10 different binary releases (2 scala versions * (4 hadoop version + hadoop-free)), which has increased the size of our release to more than 2 GB.
Naturally, building Flink 10 times also takes a while, slowing down the release process.

However, the only difference between the hadoop versions is the bundled flink-shaded-hadoop2 jar; the rest is completely identical.

I propose to stop releasing hadoop-specific distributions, and instead have us release multiple versions of flink-shaded-hadoop2 that users copy into the hadoop-free distribution if required.

Sub-Tasks

• Update create_binary_release.sh to not create hadoop-specific releases: Closed
• Update deploy_staging_jars.sh to deploy multiple flink-shaded-hadoop2 versions: Closed
• Do not include hadoop in flink-dist by default: Closed