[HADOOP-9293] For S3 use credentials file

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Affects Version/s: 1.0.2
• Component/s: fs/s3
• Labels: features newbie

Environment

Linux

• Assignee: Unassigned
• Reporter: Andy Sautins
• Created: 08/Feb/13 20:22
• Updated: 16/Jun/17 10:31
• Resolved: 16/Jun/17 10:31

Description

The following document describes the current way that S3 credentials can be specified ( http://wiki.apache.org/hadoop/AmazonS3 ). In summary they are:

• in the S3 URI.
• in the hadoop-site.xml file as
  • fs.s3.awsAccessKeyId
  • fs.s3.awsSecretAccessKey
  • fs.s3n.awsAccessKeyId
  • fs.s3n.aswSecretAccessKey

The amazon EMR tool elastic-mapreduce already provide the ability to use a credentials file ( see http://s3.amazonaws.com/awsdocs/ElasticMapReduce/latest/emr-qrc.pdf ).

I would propose that we allow roughly the same access to credentials through a credentials file that is currently provided by elastic-mapreduce. This should allow for centralized administration of credentials which should be positive for security.

I propose the following properties:

<property><name>f3.s3.awsCredentialsFile</name><value>/path/to/file</value></property>
<property><name>fs.s3n.awsCredentialsFile</name><value>/path/to/file</value></property>