[HADOOP-14661] S3A to support Requester Pays Buckets

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha3
• Fix Version/s: 3.3.5
• Component/s: common, util
• Labels: pull-request-available
• Target Version/s: 3.4.0
• Tags: aws,s3,requester pays
• Assignee: Daniel Carl Jones
• Reporter: Mandus Momberg
• Created: 14/Jul/17 03:05
• Updated: 30/Jan/24 20:02
• Resolved: 23/Mar/22 20:19

Description

Amazon S3 has the ability to charge the requester for the cost of accessing S3. This is called Requester Pays Buckets.

In order to access these buckets, each request needs to be signed with a specific header.
http://docs.aws.amazon.com/AmazonS3/latest/dev/RequesterPaysBuckets.html

Issue Links

• causes: HADOOP-18201 Remove base and bucket overrides for endpoint in ITestS3ARequesterPays.java
• depends upon: HADOOP-17511 Add an Audit plugin point for S3A auditing/context
• is related to: HADOOP-19057 S3 public test bucket landsat-pds unreadable -needs replacement
• links to: AWS SDK: Not all s3 requests in sdk provide option for setting Requester pays
• links to: GitHub Pull Request #250
• links to: GitHub Pull Request #3962