[HADOOP-18352] Support AWS IAM Identity Centre (prev. AWS SSO) for providing credentials to S3A

• Type: Sub-task
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.3.3
• Component/s: fs/s3
• Assignee: Unassigned
• Reporter: Daniel Carl Jones
• Created: 20/Jul/22 12:18
• Updated: 03/Dec/24 13:39

Description

HADOOP-18073 was opened regarding upgrading AWS SDK to V2 which supports a credential provider for AWS SSO. Opening this ticket to track that feature explicitly separately from the SDK upgrade.

Related SDK issue for AWS SSO support in AWS SDK for Java V1: https://github.com/aws/aws-sdk-java/issues/2434

In particular, from https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html, when to set 'fs.s3a.aws.credentials.provider', it must be "com.amazonaws.auth.AWSCredentialsProvider". We would like to support "software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider" which supports AWS SSO, so users only need to authenticate once.

Issue Links

• relates to: HADOOP-18073 S3A: Upgrade AWS SDK to V2