[HADOOP-16806] AWS AssumedRoleCredentialProvider needs ExternalId add

• Type: Sub-task
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.2.1
• Component/s: fs/s3
• Labels: pull-request-available
• Flags: Patch
• Assignee: Unassigned
• Reporter: Jon Hartlaub
• Created: 15/Jan/20 18:58
• Updated: 03/Dec/24 13:39

Description

AWS has added a security feature to the assume-role function in the form of the "ExternalId" key in the AWS Java SDK STSAssumeRoleSessionCredentialsProvider.Builder class.  To support this security feature, the hadoop aws AssumedRoleCredentialProvider needs a patch to include this value from the configuration as well as an added Constant to the org.apache.hadoop.fs.s3a.Constants file.

The ExternalId is not a required security feature, it is an augmentation of the current assume role configuration. 

Proposed: 

• Get the assume-role ExternalId token from the configuration for the configuration key fs.s3a.assumed.role.externalid
• Use the configured ExternalId value in the STSAssumeRoleSessionCredentialsProvider.Builder   

e.g.

if (StringUtils.isNotEmpty(externalId)) {
    builder.withExternalId(externalId); // include the token for cross-account assume role
}

 Tests:

• Unit test which verifies the ExternalId state value of the AssumedRoleCredentialProvider is consistent with the configured value - either empty or populated
• Question: not sure about how to write the integration test for this feature.  We have an account configured for this use-case that verifies this feature but I don't have much context on the Hadoop project AWS S3 integration tests, perhaps a pointer could help.

 

 

Issue Links

• links to: GitHub Pull Request #4753