Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-16829 Über-jira: S3A Hadoop 3.4 features
  3. HADOOP-16806

AWS AssumedRoleCredentialProvider needs ExternalId add

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 3.2.1
    • Fix Version/s: None
    • Component/s: fs/s3
    • Labels:
      None
    • Flags:
      Patch

      Description

      AWS has added a security feature to the assume-role function in the form of the "ExternalId" key in the AWS Java SDK STSAssumeRoleSessionCredentialsProvider.Builder class.  To support this security feature, the hadoop aws AssumedRoleCredentialProvider needs a patch to include this value from the configuration as well as an added Constant to the org.apache.hadoop.fs.s3a.Constants file.

      The ExternalId is not a required security feature, it is an augmentation of the current assume role configuration. 

      Proposed: 

      • Get the assume-role ExternalId token from the configuration for the configuration key fs.s3a.assumed.role.externalid
      • Use the configured ExternalId value in the STSAssumeRoleSessionCredentialsProvider.Builder   

      e.g.

      if (StringUtils.isNotEmpty(externalId)) {
          builder.withExternalId(externalId); // include the token for cross-account assume role
      }

       Tests:

      • Unit test which verifies the ExternalId state value of the AssumedRoleCredentialProvider is consistent with the configured value - either empty or populated
      • Question: not sure about how to write the integration test for this feature.  We have an account configured for this use-case that verifies this feature but I don't have much context on the Hadoop project AWS S3 integration tests, perhaps a pointer could help.

       

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jhartlaub Jon Hartlaub
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: