Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15220 Über-jira: S3a phase V: Hadoop 3.2 features
  3. HADOOP-14833

Remove s3a user:secret authentication

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0-beta1
    • 3.2.0
    • fs/s3, security
    • None
    • Incompatible change, Reviewed
    • Hide
      The S3A connector no longer supports username and secrets in URLs of the form `s3a://key:secret@bucket/`. It is near-impossible to stop those secrets being logged —which is why a warning has been printed since Hadoop 2.8 whenever such a URL was used.

      Fix: use a more secure mechanism to pass down the secrets.
      Show
      The S3A connector no longer supports username and secrets in URLs of the form ` s3a://key: secret@bucket /`. It is near-impossible to stop those secrets being logged —which is why a warning has been printed since Hadoop 2.8 whenever such a URL was used. Fix: use a more secure mechanism to pass down the secrets.

    Description

      Remove the s3a://user:secret@host auth mechanism from S3a.

      As well as being insecure, it causes problems with S3Guard's URI matching code.

      Proposed: cull it utterly. We've been telling people to stop using it since HADOOP-3733

      Attachments

        1. HADOOP-14833-001.patch
          43 kB
          Steve Loughran
        2. HADOOP-14833-002.patch
          45 kB
          Steve Loughran

        Issue Links

          blocks

          Sub-task - The sub-task of the issue HADOOP-14556 S3A to support Delegation Tokens

          • Major - Major loss of function.
          • Resolved
          causes

          Sub-task - The sub-task of the issue HADOOP-15750 remove obsolete S3A test ITestS3ACredentialsInURL

          • Major - Major loss of function.
          • Resolved
          contains

          Sub-task - The sub-task of the issue HADOOP-14762 S3A warning of obsolete encryption key which is never used

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Sub-task - The sub-task of the issue HADOOP-15747 warning about user:pass in URI to explicitly call out Hadoop 3.2 as removal

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-14439 regression: secret stripping from S3x URIs breaks some downstream code

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue HADOOP-13252 Tune S3A provider plugin mechanism

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-15422 s3guard doesn't init when the secrets are in the s3a URI

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-3733 "s3:" URLs break when Secret Key contains a slash, even if encoded

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. SPARK-20799 Unable to infer schema for ORC/Parquet on S3N when secrets are in the URL

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Activity

            People

              stevel@apache.org Steve Loughran
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: