Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12807

S3AFileSystem should read AWS credentials from environment variables

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.7.2
    • 2.8.0, 3.0.0-alpha1
    • fs/s3
    • None
    • Adds support to S3AFileSystem for reading AWS credentials from environment variables.

    Description

      Unlike the DefaultAWSCredentialsProviderChain in the AWS SDK, the AWSCredentialsProviderChain constructed by S3AFileSystem does not include an EnvironmentVariableCredentialsProvider instance. This prevents users from supplying AWS credentials in the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, which is the only alternative in some scenarios.

      In my scenario, I need to access S3 from within a test running in a CI environment that does not support IAM roles but does allow me to supply encrypted environment variables. Thus, the only secure approach I can use is to supply my AWS credentials in environment variables (plaintext configuration files are out of the question).

      Attachments

        1. HADOOP-12807-branch-2-004.patch
          3 kB
          Steve Loughran
        2. HADOOP-12807-1.patch
          1 kB
          Tobin Baker

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            tdbaker Tobin Baker
            tdbaker Tobin Baker
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment