[HADOOP-18938] S3A region logic to handle vpce and non standard endpoints

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.5.0
• Component/s: fs/s3
• Labels: pull-request-available
• Assignee: Shintaro Onuma
• Reporter: Ahmar Suhail
• Created: 16/Oct/23 08:16
• Updated: 05/Sep/24 19:34
• Resolved: 05/Sep/24 13:14

Description

For non standard endpoints such as VPCE the region parsing added in HADOOP-18908 doesn't work. This is expected as that logic is only meant to be used for standard endpoints. 

If you are using a non-standard endpoint, check if a region is also provided, else fail fast. 

Also update documentation to explain to region and endpoint behaviour with SDK V2. 

Issue Links

• relates to: HADOOP-18908 Improve s3a region handling, including determining from endpoint
• links to: GitHub Pull Request #6466