[HADOOP-17705] S3A to add option fs.s3a.endpoint.region to set AWS region

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.3.2
• Component/s: fs/s3
• Labels: pull-request-available
• Assignee: Mehakmeet Singh
• Reporter: Mehakmeet Singh
• Created: 17/May/21 11:21
• Updated: 25/May/21 10:57
• Resolved: 24/May/21 12:14

Description

Currently, AWS region is either constructed via the endpoint URL, by making an assumption that the 2nd component after delimiter "." is the region in endpoint URL, which doesn't work for private links and sets the default to us-east-1 thus causing authorization issue w.r.t the private link.

The option fs.s3a.endpoint.region allows this to be explicitly set

Issue Links

• is depended upon by: HADOOP-15603 S3A to support configuring various AWS S3 client extended options
• is depended upon by: HADOOP-17677 Distcp is unable to determine region with S3 PrivateLink endpoints
• links to: GitHub Pull Request #3020