Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15220 Über-jira: S3a phase V: Hadoop 3.2 features
  3. HADOOP-15201

Automatically determine region & hence S3 endpoint of buckets

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: 3.0.0
    • Fix Version/s: None
    • Component/s: fs/s3
    • Labels:
      None

      Description

      Use getBucketLocation to determine location & map to endpoint, if fs.s3a.endpoint is set to "automatic"

      S3guard added the API call String getBucketLocation(), which is used for DDB binding, We can also use this to determine the s3 endpoint, to avoid recurrent issues with auth failures related to it not being valid

      Still need to handle: buckets on third-party servers, and the inevitability that new AWS regions will be added after the Hadoop version & AWS Jar is shipped and frozen

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                stevel@apache.org Steve Loughran
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: