[HADOOP-18820] AWS SDK v2: make the v1 bridging support optional

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: fs/s3
• Labels: pull-request-available
• Target Version/s: 3.4.0
• Hadoop Flags: Incompatible change

Release Note

The v1 aws-sdk-bundle JAR has been removed; it only required for third party applications or for use of v1 SDK AWSCredentialsProvider classes. There is automatic migration of the standard providers from the v1 to v2 classes, so this is only of issue for third-party providers or if very esoteric classes in the V1 SDK are used. Consult the aws_sdk_upgrade document for details

• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 20/Jul/23 16:31
• Updated: 27/Jan/24 08:18
• Resolved: 23/Aug/23 10:33

Description

The AWS SDK v2 code includes the v1 sdk core for plugin support of

• existing credential providers
• delegation token binding

I propose we break #2 and rely on those who have implemented to to upgrade. apart from all the needless changes the v2 SDK did to the api (why?) this is fairly straighforward

for #1: fix through reflection, retaining a v1 sdk dependency at test time so we can verify that the binder works.

Issue Links

• contains: HADOOP-18812 list AWS SDK v2 libraries in LICENSE-binary
• contains: HADOOP-18743 convert declarations of AWS v1 SDK EnvironmentVariableCredentialsProvider to v2 version
• contains: HADOOP-18741 AWS SDK v2 code tuning
• contains: HADOOP-18819 AWS SDK v2 build complaints
• is depended upon by: HADOOP-18818 Merge aws v2 upgrade feature branch into trunk
• links to: GitHub Pull Request #5872
• links to: GitHub Pull Request #5973