VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.8.0
    • 2.8.0, 3.0.0-alpha1
    • fs/s3
    • None
    • Reviewed
    • Hide
      S3A now supports configuration of multiple credential provider classes for authenticating to S3. These are loaded and queried in sequence for a valid set of credentials. For more details, refer to the description of the fs.s3a.aws.credentials.provider configuration property or the S3A documentation page.
      Show
      S3A now supports configuration of multiple credential provider classes for authenticating to S3. These are loaded and queried in sequence for a valid set of credentials. For more details, refer to the description of the fs.s3a.aws.credentials.provider configuration property or the S3A documentation page.

    Description

      We've now got some fairly complex auth mechanisms going on: -hadoop config, KMS, env vars, "none". IF something isn't working, it's going to be a lot harder to debug.

      Review and tune the S3A provider point

      • add logging of what's going on in s3 auth to help debug problems
      • make a whole chain of logins expressible
      • allow the anonymous credentials to be included in the list
      • review and updated documents.

      I propose carefully adding some debug messages to identify which auth provider is doing the auth, so we can see if the env vars were kicking in, sysprops, etc.

      What we mustn't do is leak any secrets: this should be identifying whether properties and env vars are set, not what their values are. I don't believe that this will generate a security risk.

      Attachments

        1. HADOOP-13252-branch-2-001.patch
          34 kB
          Steve Loughran
        2. HADOOP-13252-branch-2-003.patch
          46 kB
          Steve Loughran
        3. HADOOP-13252-branch-2-004.patch
          46 kB
          Steve Loughran
        4. HADOOP-13252-branch-2-005.patch
          49 kB
          Steve Loughran
        5. HADOOP-13252-006.patch
          52 kB
          Steve Loughran
        6. HADOOP-13252-007.patch
          55 kB
          Steve Loughran

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            stevel@apache.org Steve Loughran
            stevel@apache.org Steve Loughran
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment