Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12807

S3AFileSystem should read AWS credentials from environment variables

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.7.2
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: fs/s3
    • Labels:
      None
    • Release Note:
      Adds support to S3AFileSystem for reading AWS credentials from environment variables.

      Description

      Unlike the DefaultAWSCredentialsProviderChain in the AWS SDK, the AWSCredentialsProviderChain constructed by S3AFileSystem does not include an EnvironmentVariableCredentialsProvider instance. This prevents users from supplying AWS credentials in the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, which is the only alternative in some scenarios.

      In my scenario, I need to access S3 from within a test running in a CI environment that does not support IAM roles but does allow me to supply encrypted environment variables. Thus, the only secure approach I can use is to supply my AWS credentials in environment variables (plaintext configuration files are out of the question).

        Attachments

        1. HADOOP-12807-branch-2-004.patch
          3 kB
          Steve Loughran
        2. HADOOP-12807-1.patch
          1 kB
          Tobin Baker

          Issue Links

            Activity

              People

              • Assignee:
                tdbaker Tobin Baker
                Reporter:
                tdbaker Tobin Baker
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: