[HADOOP-19083] provide hadoop binary tarball without aws v2 sdk

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.1
• Component/s: build, fs/s3
• Labels: pull-request-available
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 21/Feb/24 17:25
• Updated: 25/Oct/24 09:50
• Resolved: 08/Oct/24 17:45

Description

Have the default hadoop binary .tar.gz exclude the aws v2 sdk by default.

This SDK brings the total size of the distribution to about 1 GB.

Proposed

• add a profile to include the aws sdk in the dist module
• document it for local building
• for release builds, we modify our release ant builds to generate modified x86 and arm64 releases without the file.

Issue Links

• blocks: HADOOP-19087 Release Hadoop 3.4.1
• is duplicated by: HADOOP-19206 Hadoop release contains a 530MB bundle-2.23.19.jar
• links to: GitHub Pull Request #6570